etree.parse close open file object
Hello Since the version 2.3 of lxml, I have a problem with parse function of etree. When I a provide an open file object to the etree.parse function, the file object is automatically closed by parse. Is there a way to keep the file open? Thanks for help. Best regards, Bertrand Néron
bertrand neron, 01.04.2011 17:12:
Since the version 2.3 of lxml, I have a problem with parse function of etree. When I a provide an open file object to the etree.parse function, the file object is automatically closed by parse.
Is there a way to keep the file open?
That was specifically changed in order to prevent dangling open files. Would you care to give some hints about your use case? Stefan
On 04/01/2011 05:49 PM, Stefan Behnel wrote:
bertrand neron, 01.04.2011 17:12:
Since the version 2.3 of lxml, I have a problem with parse function of etree. When I a provide an open file object to the etree.parse function, the file object is automatically closed by parse.
Is there a way to keep the file open?
That was specifically changed in order to prevent dangling open files.
Would you care to give some hints about your use case?
Stefan
I have concurrent access to the xml file I must parse. So after opening file I acquire a lock on this file. I parse the xml, then modify it and serialize it before release the lock on the file. but now I cannot use lock anymore. I'd like the parse function does not modify the open/close status of the file given as argument. bertrand
On 04/01/2011 05:49 PM, Stefan Behnel wrote:
bertrand neron, 01.04.2011 17:12:
Since the version 2.3 of lxml, I have a problem with parse function of etree. When I a provide an open file object to the etree.parse function, the file object is automatically closed by parse.
Is there a way to keep the file open? That was specifically changed in order to prevent dangling open files.
Would you care to give some hints about your use case?
Stefan I have concurrent access to the xml file I must parse. So after opening file I acquire a lock on this file. I parse the xml, then modify it and serialize it before release the lock on the file.
but now I cannot use lock anymore. I'd like the parse function does not modify the open/close status of the file given as argument.
bertrand I second the opinion of Bertrand. Actually, IMHO, the problem is that
Hi, On 04/04/2011 09:52 AM, bertrand neron wrote: the "parse" method processes either a file path or a file handle: in the case where we provide a file path, the file should of course be closed after being read, but in the case where we provide a file handle, I think it is the responsibility of the "caller code" to close the file. By the way, thank you very much to all of the author(s) of lxml, it's a really performant library, switching to it really gained us much performance. Hervé
_________________________________________________________________ Mailing list for the lxml Python XML toolkit - http://lxml.de/ lxml@lxml.de http://mailman-mail5.webfaction.com/listinfo/lxml
Hervé Ménager, 23.05.2011 13:12:
On 04/04/2011 09:52 AM, bertrand neron wrote:
On 04/01/2011 05:49 PM, Stefan Behnel wrote:
bertrand neron, 01.04.2011 17:12:
Since the version 2.3 of lxml, I have a problem with parse function of etree. When I a provide an open file object to the etree.parse function, the file object is automatically closed by parse.
Is there a way to keep the file open? That was specifically changed in order to prevent dangling open files.
Would you care to give some hints about your use case?
I have concurrent access to the xml file I must parse. So after opening file I acquire a lock on this file. I parse the xml, then modify it and serialize it before release the lock on the file.
but now I cannot use lock anymore. I'd like the parse function does not modify the open/close status of the file given as argument.
I second the opinion of Bertrand. Actually, IMHO, the problem is that the "parse" method processes either a file path or a file handle: in the case where we provide a file path, the file should of course be closed after being read, but in the case where we provide a file handle, I think it is the responsibility of the "caller code" to close the file.
Actually, it's even worse. The resolvers mechanism allows the user to return a file-like object from a callback, so it's unlikely that the user keeps a reference to it for future cleanup. Should this be closed or not? I guess it would be best to provide an option in that case, defaulting to auto-close the file. Apart from that, I agree with your above comments.
By the way, thank you very much to all of the author(s) of lxml, it's a really performant library, switching to it really gained us much performance.
:) Stefan
On 05/23/2011 02:09 PM, Stefan Behnel wrote:
Hervé Ménager, 23.05.2011 13:12:
On 04/04/2011 09:52 AM, bertrand neron wrote:
On 04/01/2011 05:49 PM, Stefan Behnel wrote:
bertrand neron, 01.04.2011 17:12:
Since the version 2.3 of lxml, I have a problem with parse function of etree. When I a provide an open file object to the etree.parse function, the file object is automatically closed by parse.
Is there a way to keep the file open? That was specifically changed in order to prevent dangling open files.
Would you care to give some hints about your use case?
I have concurrent access to the xml file I must parse. So after opening file I acquire a lock on this file. I parse the xml, then modify it and serialize it before release the lock on the file.
but now I cannot use lock anymore. I'd like the parse function does not modify the open/close status of the file given as argument.
I second the opinion of Bertrand. Actually, IMHO, the problem is that the "parse" method processes either a file path or a file handle: in the case where we provide a file path, the file should of course be closed after being read, but in the case where we provide a file handle, I think it is the responsibility of the "caller code" to close the file.
Actually, it's even worse. The resolvers mechanism allows the user to return a file-like object from a callback, so it's unlikely that the user keeps a reference to it for future cleanup. Should this be closed or not?
Mmm... Then I guess I had not understood all the complexity of the lxml usecases (this does not come as a surprise, though). For such complex cases, I cannot claim to be an expert.
I guess it would be best to provide an option in that case, defaulting to auto-close the file. Apart from that, I agree with your above comments.
In our case, getting this option would clearly solve the issue. I would be happy to see it in lxml!
By the way, thank you very much to all of the author(s) of lxml, it's a really performant library, switching to it really gained us much performance.
:)
Stefan
Hervé Ménager, 23.05.2011 14:51:
On 05/23/2011 02:09 PM, Stefan Behnel wrote:
Hervé Ménager, 23.05.2011 13:12:
On 04/04/2011 09:52 AM, bertrand neron wrote:
On 04/01/2011 05:49 PM, Stefan Behnel wrote:
bertrand neron, 01.04.2011 17:12:
Since the version 2.3 of lxml, I have a problem with parse function of etree. When I a provide an open file object to the etree.parse function, the file object is automatically closed by parse.
Is there a way to keep the file open? That was specifically changed in order to prevent dangling open files.
Would you care to give some hints about your use case?
I have concurrent access to the xml file I must parse. So after opening file I acquire a lock on this file. I parse the xml, then modify it and serialize it before release the lock on the file.
but now I cannot use lock anymore. I'd like the parse function does not modify the open/close status of the file given as argument.
I second the opinion of Bertrand. Actually, IMHO, the problem is that the "parse" method processes either a file path or a file handle: in the case where we provide a file path, the file should of course be closed after being read, but in the case where we provide a file handle, I think it is the responsibility of the "caller code" to close the file.
Actually, it's even worse. The resolvers mechanism allows the user to return a file-like object from a callback, so it's unlikely that the user keeps a reference to it for future cleanup. Should this be closed or not?
Mmm... Then I guess I had not understood all the complexity of the lxml usecases (this does not come as a surprise, though). For such complex cases, I cannot claim to be an expert.
I guess it would be best to provide an option in that case, defaulting to auto-close the file. Apart from that, I agree with your above comments.
In our case, getting this option would clearly solve the issue. I would be happy to see it in lxml!
https://github.com/lxml/lxml/commit/48f115fa5d2a515d70d0428515be7d9b5d1bff9a Stefan
On 05/23/2011 08:51 PM, Stefan Behnel wrote:
Hervé Ménager, 23.05.2011 14:51:
On 05/23/2011 02:09 PM, Stefan Behnel wrote:
Hervé Ménager, 23.05.2011 13:12:
On 04/04/2011 09:52 AM, bertrand neron wrote:
On 04/01/2011 05:49 PM, Stefan Behnel wrote:
bertrand neron, 01.04.2011 17:12: > Since the version 2.3 of lxml, I have a problem with parse > function of > etree. When I a provide an open file object to the etree.parse > function, > the file object is automatically closed by parse. > > Is there a way to keep the file open? That was specifically changed in order to prevent dangling open files.
Would you care to give some hints about your use case?
I have concurrent access to the xml file I must parse. So after opening file I acquire a lock on this file. I parse the xml, then modify it and serialize it before release the lock on the file.
but now I cannot use lock anymore. I'd like the parse function does not modify the open/close status of the file given as argument.
I second the opinion of Bertrand. Actually, IMHO, the problem is that the "parse" method processes either a file path or a file handle: in the case where we provide a file path, the file should of course be closed after being read, but in the case where we provide a file handle, I think it is the responsibility of the "caller code" to close the file.
Actually, it's even worse. The resolvers mechanism allows the user to return a file-like object from a callback, so it's unlikely that the user keeps a reference to it for future cleanup. Should this be closed or not?
Mmm... Then I guess I had not understood all the complexity of the lxml usecases (this does not come as a surprise, though). For such complex cases, I cannot claim to be an expert.
I guess it would be best to provide an option in that case, defaulting to auto-close the file. Apart from that, I agree with your above comments.
In our case, getting this option would clearly solve the issue. I would be happy to see it in lxml!
https://github.com/lxml/lxml/commit/48f115fa5d2a515d70d0428515be7d9b5d1bff9a
Stefan
Thank you very much Stefan, I will test this ASAP. Best, Hervé
participants (3)
-
bertrand neron -
Hervé Ménager -
Stefan Behnel