[GET-dev] trait-o-matic's mysql queries seem unnecessarily slow

[Alexander Wait Zaranek]

On Thu, Sep 16, 2010 at 2:55 PM, Madeleine Price Ball <meprice at gmail.com> wrote:
> I suspect all the other steps in Trait-o-matic processing can be improved in
> the same way: simultaneously moving through pre-sorted files rather than
> loading one into MySQL and then querying MySQL.
I wrote code for another project that does exactly this.  In that
project, UNIX sort / IO / compression was the bottleneck for
data-processing.

We might be able to do better than pre-sorting but, clearly, it's an
improvement over the existing method.

Sasha