[Baypiggies] I need some help architecting the big picture

Shannon -jj Behrens jjinux at gmail.com
Tue Apr 29 07:04:12 CEST 2008

On Mon, Apr 28, 2008 at 3:02 PM, Michael Pittaro <mikeyp at snaplogic.org> wrote:
> Shannon -jj Behrens wrote:
> > Hi,
> >
> > I need some help architecting the big picture on my current project.
> > I'm usually a Web guy, which I understand very well.  However, my
> > current project is more batch oriented.  Here are the details:
> > Anyway, so I have all this customer-specifc logic, and all these data
> > pipelines.  How do I pull it together into something an operator would
> > want to use?  Is the idea of an operator appropriate?  I'm pretty sure
> > this is an "operations" problem.
> >
> >
>  JJ,
>  This sounds like a typical data integration project, which is what
> SnapLogic is designed for.
>  Depending on what sort of operations you're doing on the data, we might
> already have the right components or you might need to create a new one, but
> from your description,  it sounds like we have most of the database and file
> IO you need.
>  You're right about the operations aspect - you probably want an 'operator'
> interface that that will just run everything when the right files are in
> place, either scheduled or triggered by some event.
>  Drop me a line if you need more details.

Ah, I haven't forgotten about SnapLogic.  I'm not sure I completely
know the definition of a data integration project.  I do know I have
data going in.  The key thing that I'm doing is apply clever
algorithms to learn stuff from the data.  Then I can use the stuff I
learned to answer queries via a Web interface.


I, for one, welcome our new Facebook overlords!

More information about the Baypiggies mailing list