Kevin, Is there a question there? I'd be happy to try to help if I knew what you were asking. On Thu, Aug 7, 2014 at 11:42 PM, Kevin Mcintyre <kebin70@gmail.com> wrote:

wordsmith argument order issues re-take.

Cheers, Kevin

On Thu, Aug 7, 2014 at 11:29 PM, Kevin Mcintyre <kebin70@gmail.com> wrote:

...

goal: Give knowitall.py awareness of wordsmith.py addition.

to run ./library.sh # opens 7999 for remote_dictionary and remote_define methods then both: python knowitall.py python wordsmith.py

library.sh due to main/jelly issues

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

-- Daniel Sank Department of Physics Broida Hall University of California Santa Barbara, CA 93117 (805)893-3899

I think Cacheable exists to handle the case you want. Beware there are some funny issues with Cacheable https://twistedmatrix.com/trac/ticket/7274 Daniel On Fri, Aug 8, 2014 at 8:01 AM, Kevin Mcintyre <kebin70@gmail.com> wrote:

The question is what's necessary to get knowitall.py to receive updates to held copy.

I start server, start knowitall, run wordsmith. Knowitall doesn't receive updates. I understand why, setCopyableState only gets fired once. Guessing jelly doesn't hold onto the RemoteCopy or a reference to RemoteCopy.

Is the next step to add pb.Referenceable to Dictionary class, and use observe_?

On Fri, Aug 8, 2014 at 12:30 AM, Daniel Sank <sank.daniel@gmail.com> wrote:

...

Kevin,

Is there a question there? I'd be happy to try to help if I knew what you were asking.

On Thu, Aug 7, 2014 at 11:42 PM, Kevin Mcintyre <kebin70@gmail.com> wrote:

...

wordsmith argument order issues re-take.

Cheers, Kevin

On Thu, Aug 7, 2014 at 11:29 PM, Kevin Mcintyre <kebin70@gmail.com> wrote:

...

goal: Give knowitall.py awareness of wordsmith.py addition.

to run ./library.sh # opens 7999 for remote_dictionary and remote_define methods then both: python knowitall.py python wordsmith.py

library.sh due to main/jelly issues

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

-- Daniel Sank Department of Physics Broida Hall University of California Santa Barbara, CA 93117 (805)893-3899

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

-- Daniel Sank Department of Physics Broida Hall University of California Santa Barbara, CA 93117 (805)893-3899

On Aug 8, 2014, at 1:31 PM, Kevin Mcintyre <kebin70@gmail.com> wrote:

I think so too, but I'm starting to lean toward websockets. The warnings alone right?

Websockets are a layer well below what PB does. In fact you could run PB over websockets, and it would work roughly the same as over TCP.

But I will take another pass and report back any success.

Please do let us know. -glyph

correction. I was stupidly reprocessing list on remote copy side. Seeing much better results now ~10K per second. On Fri, Aug 8, 2014 at 6:32 PM, Kevin Mcintyre <kebin70@gmail.com> wrote:

I experimented today with a table scan from dynamodb. Taking each record and converting to a dictionary into a list. Initial scan and transform took ~8 seconds from amazon to my house over TCP boto which is soapish? i think. ~7800 records.

The next call for the remote copy took 7 seconds across the bus. Is that expected?

On Fri, Aug 8, 2014 at 3:01 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:

...

On Aug 8, 2014, at 1:31 PM, Kevin Mcintyre <kebin70@gmail.com> wrote:

I think so too, but I'm starting to lean toward websockets. The warnings alone right?

Websockets are a layer well below what PB does. In fact you could run PB over websockets, and it would work roughly the same as over TCP.

But I will take another pass and report back any success.

Please do let us know.

-glyph

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Jelly performance factors below expectations. Can we say Copyable is the lowest order jelly? The notion that a copy holder can't ask "is my copy good anymore?" makes it so. Essentially root says, I'd prefer not to repeat unit of work nor keep track of the resulting copies, here have the original or resulting copy. My main issue is a copy-holder calling for a copy to determine is the copy is good anymore. I know, see cacheable but it's problematic. At this point I'm unsure of what jelly actually does well. who's the JellyCon2014 keynote speaker? On Sat, Aug 9, 2014 at 6:58 PM, Glyph <glyph@twistedmatrix.com> wrote:

On Aug 8, 2014, at 9:17 PM, Kevin Mcintyre <kebin70@gmail.com> wrote:

correction. I was stupidly reprocessing list on remote copy side. Seeing much better results now ~10K per second.

Whew. I don't know exactly how well I would expect this perform, but those other results looked off by a couple orders of magnitude :-).

-glyph

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Jelly performance factors below expectations. I am not clear on the significance of this statement. Not significant, just an observation. I've made this statement before, backed-off after finding errors on my code, but after fixing I see performance across the bus that still seem slow. "Seem slow" lacks any scientific backing. I would be curious to hear use cases for PB/Jelly that went beyond the docs. Can we say Copyable is the lowest order jelly? The notion that a copy holder can't ask "is my copy good anymore?" makes it so. Essentially root says, I'd prefer not to repeat unit of work nor keep track of the resulting copies, here have the original or resulting copy. Asking whether your copy is good any more is a PB-level task. Jelly itself is a separate layer which is about getting the right data to the right place, not keeping it updated. Yes PB level, I guess I'm looking for a convention where 1 does not exist. In the renewed interest the comments have tended to overlook copyable, or see copyable as being flushed out. My main issue is a copy-holder calling for a copy to determine is the copy is good anymore. I know, see cacheable but it's problematic. "problematic" is passive voice :-). What are the problems? Only parroting what I've read on cacheable, haven't delved into yet. At this point I'm unsure of what jelly actually does well. Malheureusement, Jelly's main claim to fame is "it's better than pickle". But with all this renewed interest in PB perhaps we'll get pre-deserialization schema enforcement and type checking, and then it will have some real advantages :). Agreed. This is what I looked to Jelly/PB for, having found so much in twisted that simplifies. who's the JellyCon2014 keynote speaker? *One* day we will have a conference - Tx/Rx, of course - and there will be a talk on this. But first we need several people in the community who wants their contribution to Twisted to be organization and community oriented rather than code ;-). On Sun, Sep 7, 2014 at 9:43 AM, Glyph <glyph@twistedmatrix.com> wrote:

On Sep 7, 2014, at 12:51 AM, Kevin Mcintyre <kebin70@gmail.com> wrote:

Jelly performance factors below expectations.

I am not clear on the significance of this statement.

Can we say Copyable is the lowest order jelly? The notion that a copy holder can't ask "is my copy good anymore?" makes it so. Essentially root says, I'd prefer not to repeat unit of work nor keep track of the resulting copies, here have the original or resulting copy.

Asking whether your copy is good any more is a PB-level task. Jelly itself is a separate layer which is about getting the right data to the right place, not keeping it updated.

My main issue is a copy-holder calling for a copy to determine is the copy is good anymore. I know, see cacheable but it's problematic.

"problematic" is passive voice :-). What are the problems?

At this point I'm unsure of what jelly actually does well.

Malheureusement, Jelly's main claim to fame is "it's better than pickle". But with all this renewed interest in PB perhaps we'll get pre-deserialization schema enforcement and type checking, and then it will have some real advantages :).

who's the JellyCon2014 keynote speaker?

*One* day we will have a conference - Tx/Rx, of course - and there will be a talk on this. But first we need several people in the community who wants their contribution to Twisted to be organization and community oriented rather than code ;-).

-glyph

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python