[Twisted-Python] Media Resource Control Protocol (MRCP)
Dear All I am interested in writing an MRCP (Media Resource Control Protocol [1]) server. In particular I want to write an MRCP speech recognition resource server as part of my trefnydd [2] project. I am posting here to ask for advice about the network programming part. I shall also be writing an MRCP client (if only to test the server). All the code will be GPL'd. As far as I understand it, an overview of the networking would go like this: An MRCP session between client and server is initiated using the Session Initiation Protocol (SIP). Client and server negotiate using SIP messages containing SDP message bodies to define the participants and the resources required in the session. After session initiation, two channels are launched: a media channel using RTP to carry binary audio (i.e., speech) data, and a control channel using TCP (control channel messages are text-based and in MRCP format). It seems to me that both client and server software would have to cope with SIP, SDP, RTP, MRCP (message format) and plain old TCP (for the SIP/SDP and MRCP control channel messages). Here are my questions: - Does anyone know of a FOSS project to write an MRCP server? preferably in python. I know of mrcp4j [3]. - My first plan would be to make the MRCP client/server a subclass of an SIP client/server, as SIP is used to set up (and tear down) the session. Does that sound sensible? - Can twisted.protocols.sip be used to write a SIP server? Would it give me everything I need, or would I also need something like pjsip [4] or jain-sip [5] for the low-level stuff? - Have I missed anything? Any advice would be gratefully received. I'll report back with progress. Best wishes Ivan [1] MRCP: http://www.llaisdy.com/static/tech/mrcp/ [2] trefnydd: http://www.llaisdy.com/static/trefnydd/ [3] mrcp4j: "MRCP4J provides a Java API that encapsulates the MRCPv2 protocol and can be used to implement MRCPv2 clients and/or servers." http://mrcp4j.sourceforge.net/ [4] pjsip: http://www.pjsip.org/ [5] jain-sip: https://jain-sip.dev.java.net/
You may wish to look at divmod's sine project that provides an sip server. Why not license under MIT so that if this is successful it may become another a supported protocol in the twisted stack. Regards, David Ivan Uemlianin wrote:
Dear All
I am interested in writing an MRCP (Media Resource Control Protocol [1]) server. In particular I want to write an MRCP speech recognition resource server as part of my trefnydd [2] project. I am posting here to ask for advice about the network programming part.
I shall also be writing an MRCP client (if only to test the server). All the code will be GPL'd.
As far as I understand it, an overview of the networking would go like this:
An MRCP session between client and server is initiated using the Session Initiation Protocol (SIP). Client and server negotiate using SIP messages containing SDP message bodies to define the participants and the resources required in the session. After session initiation, two channels are launched: a media channel using RTP to carry binary audio (i.e., speech) data, and a control channel using TCP (control channel messages are text-based and in MRCP format).
It seems to me that both client and server software would have to cope with SIP, SDP, RTP, MRCP (message format) and plain old TCP (for the SIP/SDP and MRCP control channel messages).
Here are my questions:
- Does anyone know of a FOSS project to write an MRCP server? preferably in python. I know of mrcp4j [3]. - My first plan would be to make the MRCP client/server a subclass of an SIP client/server, as SIP is used to set up (and tear down) the session. Does that sound sensible? - Can twisted.protocols.sip be used to write a SIP server? Would it give me everything I need, or would I also need something like pjsip [4] or jain-sip [5] for the low-level stuff? - Have I missed anything?
Any advice would be gratefully received. I'll report back with progress.
Best wishes
Ivan
[1] MRCP: http://www.llaisdy.com/static/tech/mrcp/ [2] trefnydd: http://www.llaisdy.com/static/trefnydd/ [3] mrcp4j: "MRCP4J provides a Java API that encapsulates the MRCPv2 protocol and can be used to implement MRCPv2 clients and/or servers." http://mrcp4j.sourceforge.net/ [4] pjsip: http://www.pjsip.org/ [5] jain-sip: https://jain-sip.dev.java.net/
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Sun, 20 May 2007 16:15:44 +0100, Ivan Uemlianin
David Pratt wrote:
You may wish to look at divmod's sine project that provides an sip server. Why not license under MIT so that if this is successful it may become another a supported protocol in the twisted stack.
David
Thanks for your comments.
What does Divmod Sine provide that twisted.protocols.sip doesn't? For example, see this ticket:
http://twistedmatrix.com/trac/ticket/2444
Does that imply that twisted.protocols.sip can now (or soon) be used instead of Sine?
The Divmod Sine SIP implementation has numerous bug fixes not present in twisted.protocols.sip. Once #2444 is resolved, the protocol support in Twisted will be as good as the protocol support in Divmod Sine. What Sine will still offer above the code in Twisted is the higher level APIs which make call setup easy enough so that novel applications can focus on their novelty instead of on SIP details. Jean-Paul
David Pratt wrote:
You may wish to look at divmod's sine project that provides an sip server. Why not license under MIT so that if this is successful it may become another a supported protocol in the twisted stack.
David Thanks for your comments. What does Divmod Sine provide that twisted.protocols.sip doesn't? For example, see this ticket: http://twistedmatrix.com/trac/ticket/2444 Does that imply that twisted.protocols.sip can now (or soon) be used instead of Sine? If not, what is the difference between twisted's sip support and sine's? And has anyone tried pjsip? Best Ivan
The Divmod Sine SIP implementation has numerous bug fixes not present in twisted.protocols.sip. Once #2444 is resolved, the protocol support in Twisted will be as good as the protocol support in Divmod Sine. What Sine will still offer above the code in Twisted is the higher level APIs which make call setup easy enough so that novel applications can focus on their novelty instead of on SIP details. This sounds good. Thanks very much. I'll look into Sine (and the Shtoom
Jean-Paul Calderone wrote: phone) and report back with progress. Best Ivan
Jean-Paul Calderone wrote:
The Divmod Sine SIP implementation has numerous bug fixes not present in twisted.protocols.sip. Once #2444 is resolved, the protocol support in Twisted will be as good as the protocol support in Divmod Sine. What Sine will still offer above the code in Twisted is the higher level APIs which make call setup easy enough so that novel applications can focus on their novelty instead of on SIP details. I have attempted and failed to install Divmod Sine. I posted the message below to divmod-dev a few days ago but have yet to receive a reply (it's a quiet list). I am re-posting the message here in good faith and in the belief that it is not entirely off-topic or plain stupid.
I'd really like to start writing an MRCP server, and if it's good enough to contribute it to Twisted, so any help would be much appreciated. Best wishes Ivan Dear All I am installing the Divmod frameworks, with the intention of experimenting with Sine and Shtoom. I have installed everything except Sine with no problems (using python setup.py install). With Sine I get the following error: File "/usr/lib/python2.4/site-packages/xmantissa/plugins/sineoff.py", line 3, in ? from xmantissa import website, offering, provisioning exceptions.ImportError: cannot import name provisioning I have tried this with both the stable (0.6.1) and svn versions of Mantissa. I have grepped through all of Divmod's frameworks (inc. Shtoom) and not found provisioning except in import statements. The only class used from provisioning is BenefactorFactory. Please could someone let me know either: - Where I can get a version of xmantissa with provisioning and BenefactorFactory, or - Whether and how I can comment out the offending lines from Sine. Thanks Best wishes Ivan
Le vendredi 25 mai 2007 à 09:51 +0100, Ivan Uemlianin a écrit :
Jean-Paul Calderone wrote:
The Divmod Sine SIP implementation has numerous bug fixes not present in twisted.protocols.sip. Once #2444 is resolved, the protocol support in Twisted will be as good as the protocol support in Divmod Sine. What Sine will still offer above the code in Twisted is the higher level APIs which make call setup easy enough so that novel applications can focus on their novelty instead of on SIP details. I have attempted and failed to install Divmod Sine. I posted the message below to divmod-dev a few days ago but have yet to receive a reply (it's a quiet list). I am re-posting the message here in good faith and in the belief that it is not entirely off-topic or plain stupid.
Just my 2 cents... The Sine author looks dead for all purposes. He doesn't answer bug reports for his own code. After playing around with the SIP support in Twisted, finding a few bugs whose correction (*) was refused on the grounds that the author had forked his own code as a separate project, we at Wengo have written our own SIP stack on top of a patched twisted.protocols.sip. (*) http://twistedmatrix.com/trac/ticket/2220 Good luck :) Regards Antoine.
On Wed, May 30, 2007 at 12:32:39PM +0200, Antoine Pitrou wrote:
Just my 2 cents... The Sine author looks dead for all purposes. He doesn't answer bug reports for his own code.
Hi. I'm not dead. I'd be happy to look at any bug reports you want to file for Sine -- its bug tracker can be found at http://divmod.org/.
Dear All On divmod-dev, Jean-Paul Calderone advised me to use the latest tagged releases from http://divmod.org/trac/browser/tags/releases/ (i.e., svn co http://divmod.org:81/svn/Divmod/tags/releases/etc). I have now been able to install Sine and Mantissa. nb: I had to install Sine first - Mantissa would not install before Sine. Still heading otwards a look at sip as a basis for mrcp;) Best wishes Ivan Ivan Uemlianin wrote:
Jean-Paul Calderone wrote:
The Divmod Sine SIP implementation has numerous bug fixes not present in twisted.protocols.sip. Once #2444 is resolved, the protocol support in Twisted will be as good as the protocol support in Divmod Sine. What Sine will still offer above the code in Twisted is the higher level APIs which make call setup easy enough so that novel applications can focus on their novelty instead of on SIP details. I have attempted and failed to install Divmod Sine. I posted the message below to divmod-dev a few days ago but have yet to receive a reply (it's a quiet list). I am re-posting the message here in good faith and in the belief that it is not entirely off-topic or plain stupid. I'd really like to start writing an MRCP server, and if it's good enough to contribute it to Twisted, so any help would be much appreciated.
Best wishes
Ivan
Dear All
I am installing the Divmod frameworks, with the intention of experimenting with Sine and Shtoom. I have installed everything except Sine with no problems (using python setup.py install).
With Sine I get the following error:
File "/usr/lib/python2.4/site-packages/xmantissa/plugins/sineoff.py", line 3, in ? from xmantissa import website, offering, provisioning exceptions.ImportError: cannot import name provisioning
I have tried this with both the stable (0.6.1) and svn versions of Mantissa.
I have grepped through all of Divmod's frameworks (inc. Shtoom) and not found provisioning except in import statements. The only class used from provisioning is BenefactorFactory.
Please could someone let me know either: - Where I can get a version of xmantissa with provisioning and BenefactorFactory, or - Whether and how I can comment out the offending lines from Sine.
Thanks
Best wishes
Ivan
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Antoine Pitrou wrote:
After playing around with the SIP support in Twisted, finding a few bugs whose correction (*) was refused on the grounds that the author had forked his own code as a separate project, we at Wengo have written our own SIP stack on top of a patched twisted.protocols.sip.
(*) http://twistedmatrix.com/trac/ticket/2220 Thank for the lead. I'll check out http://www.openwengo.com/
Ivan
[ Ivan Uemlianin ]: ----------------------------- |Dear All | |I am interested in writing an MRCP (Media Resource Control Protocol |[1]) server. Great. I did that for a Brazilian company. It was not a full MRCP server, just enough to cope with their TTS needs. But I will be glad to give advice and help. | In particular I want to write an MRCP speech recognition |resource server as part of my trefnydd [2] project. That is about the opposite of what I have explored (synthesis) ;o) | I am posting here to ask for advice about the network programming part. My MRCP server followed [1], and was based on RTSP for session handling, instead of SIP. |I shall also be writing an MRCP client (if only to test the server). |All the code will be GPL'd. That is great. I hope I can contribute with that. The project we (me and João Chaves Jr) have worked on was proprietary software. That was an imposition from the client, and a source of great frustration for me. :o( |As far as I understand it, an overview of the networking would go like |this: | |An MRCP session between client and server is initiated using the |Session Initiation Protocol (SIP). The version I worked on, was based in RTSP not SIP. From the spec: """ This protocol is designed to work with streaming protocols like RTSP (Real Time Streaming Protocol) or SIP (Session Initiation Protocol) """ So if you need a MRCP server that is SIP-based for doing ASR, then I cannot help much since I have experience with a RTSP-based for doing TTS ;o) That does not mean I won't give advice if I have any worth giving! |Client and server negotiate using |SIP messages containing SDP message bodies to define the participants |and the resources required in the session. Correct. |After session initiation, two channels are launched: a media channel | using RTP to carry binary audio (i.e., speech) data, Right. I used a wrapper over JRTPLIB to handle RTP and RTCP. | and a control channel using TCP (control |channel messages are text-based and in MRCP format). Well, the RTSP-based verison used the same RTSP channel to tunnel session initiation (pure RTSP) and media control (MRCP payloads embedded in RTSP announce messages). I do not know how this is handled with SIP. |It seems to me that both client and server software would have to cope |with SIP, SDP, RTP, MRCP (message format) and plain old TCP (for the |SIP/SDP and MRCP control channel messages). Sounds about right. |Here are my questions: | |- Does anyone know of a FOSS project to write an MRCP server? |preferably in python. I know of mrcp4j [3]. Not that I know of. |- My first plan would be to make the MRCP client/server a subclass of |an SIP client/server, as SIP is used to set up (and tear down) the |session. Does that sound sensible? I used Twisted HTTP handling implementation as a starting point for the RTSP implementation since both protocols are very much alike. I wish you luck and success. best regards, Rod Senra [1] http://tools.ietf.org/html/rfc4463 ---- Rodrigo Dias Arruda Senra http://rodrigo.senra.nom.br
participants (6)
-
Antoine Pitrou -
David Pratt -
Ivan Uemlianin -
Jean-Paul Calderone -
Rodrigo Senra -
washort＠twistedmatrix.com