[Tfug] MySql and rsync

[Stephen Hooper]

On 4/17/07, Terence Rudkin <trudkin at gmail.com> wrote:
> Any MySQL experts out there?
> My company has a program using rsync to load files into a mysql database.
> It appears to mostly work, but some files are not getting inserted.   The
> process is loading about 5 records a second / 16k hour, not sure how big  a
> record is.
> Anyone with expertise is using rsync with mysql?   Or even Mysql tuning
>

Hi Terence,

Can you be more specific about what types of files you are rsyncing?
Are they flat files?  Files containing formatted data (like in a CSV
form)?

>From your description, I am imagining files that are being saved in
one place (say JPG's from a web application), and then rsynced to a
server with a MySQL database where something comes by, picks them up,
and does an SQL insert.  Am I close?

How about files with a bunch of outgoing phone calls separated by
tabs, and dumped by a PBX every night, then rsynced to a server with a
MySQL database where something comes by, picks them up, reads them in,
breaks out the appropriate fields, and makes a bunch of SQL inserts.
Am I closer with that one?

Anyways, I would say that whatever answer you get depends a lot on the
types of files... (but, in any case I am not a MySQL expert :) ).