what do you think about extending the existing assert representation with this https://pypi.python.org/pypi/datadiff would be nice if we could extend datadiff with what pytest does for string diff and then just use datadiff for all assertions because for now, it's not too useful to see a comparison of 2 dicts in the pytest it's not readable especially if 2 dicts are big and nowadays, json so dicts are everywhere datadiff is cool at showing the diff between 2 dicts the reason why im asking without the PR is the politics about the dependencies as i know you're not so positive about adding them also this standalone mode is a bit tricky in that case, i guess example: def test_ticket_filter_on_order_uid__in(app, first_ticket, second_ticket, second_basket_ticket, order, second_order): """Test if the right result is returned when filtering on basket.""" ticket_filter = pta.ticket.filter(api_key=app.uid, order_uid__in=[order['uid']]).content
assert ticket_filter == [first_ticket, second_ticket] E assert [{'code': '91...dulum'}, ...}] == [{'basket_uid'... 'EUR'}, ...}] E At index 0 diff: {'code': '911338060856', 'subtitle': {'en': u'Subtitle1 Kaleidoscope'}, 'uid': u'37bc3c15e66a45e9bf3b52233fcdff02', 'order_uid': u'08dec1bd7ec249109729de448ffcc571', 'price': {'currency': 'EUR', 'amount': '1.20'}, 'fees': [{'price': {'currency': 'EUR', 'amount': '2.00'}, 'type': 'service costs'}], 'product_uid': u'32c2a6f98f0642b9843a8dcce6dead56', 'name': {'en': u'Ticket1 \u2603 Weapon'}} != {'subtitle': {'en': u'Subtitle1 Kaleidoscope'}, 'uid': u'37bc3c15e66a45e9bf3b52233fcdff02', 'price': {'currency': 'EUR', 'amount': '1.20'}, 'fees': [{'price': {'currency': 'EUR', 'amount': '2.00'}, 'type': 'service costs'}], 'basket_uid': u'871e829da04e4cd9a45d244f55824c83', 'product_uid': u'32c2a6f98f0642b9843a8dcce6dead56', 'name': {'en': u'Ticket1 \u2603 Weapon'}} just unreadable
-- Anatoly Bubenkov
Hi, Perhaps this could a plugin (pytest-datadiff)? I know you can customize assertion messages for other types (see http://pytest.org/latest/assert.html#defining-your-own-assertion-comparison), but I don't know how that plays with assertions already built in pytest. Cheers, On Tue, Jul 29, 2014 at 5:24 AM, Anatoly Bubenkov <bubenkoff@gmail.com> wrote:
what do you think about extending the existing assert representation with this https://pypi.python.org/pypi/datadiff would be nice if we could extend datadiff with what pytest does for string diff and then just use datadiff for all assertions because for now, it's not too useful to see a comparison of 2 dicts in the pytest it's not readable especially if 2 dicts are big and nowadays, json so dicts are everywhere datadiff is cool at showing the diff between 2 dicts the reason why im asking without the PR is the politics about the dependencies as i know you're not so positive about adding them also this standalone mode is a bit tricky in that case, i guess example:
def test_ticket_filter_on_order_uid__in(app, first_ticket, second_ticket, second_basket_ticket, order, second_order): """Test if the right result is returned when filtering on basket.""" ticket_filter = pta.ticket.filter(api_key=app.uid, order_uid__in=[order['uid']]).content
assert ticket_filter == [first_ticket, second_ticket] E assert [{'code': '91...dulum'}, ...}] == [{'basket_uid'... 'EUR'}, ...}] E At index 0 diff: {'code': '911338060856', 'subtitle': {'en': u'Subtitle1 Kaleidoscope'}, 'uid': u'37bc3c15e66a45e9bf3b52233fcdff02', 'order_uid': u'08dec1bd7ec249109729de448ffcc571', 'price': {'currency': 'EUR', 'amount': '1.20'}, 'fees': [{'price': {'currency': 'EUR', 'amount': '2.00'}, 'type': 'service costs'}], 'product_uid': u'32c2a6f98f0642b9843a8dcce6dead56', 'name': {'en': u'Ticket1 \u2603 Weapon'}} != {'subtitle': {'en': u'Subtitle1 Kaleidoscope'}, 'uid': u'37bc3c15e66a45e9bf3b52233fcdff02', 'price': {'currency': 'EUR', 'amount': '1.20'}, 'fees': [{'price': {'currency': 'EUR', 'amount': '2.00'}, 'type': 'service costs'}], 'basket_uid': u'871e829da04e4cd9a45d244f55824c83', 'product_uid': u'32c2a6f98f0642b9843a8dcce6dead56', 'name': {'en': u'Ticket1 \u2603 Weapon'}} just unreadable
-- Anatoly Bubenkov
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
Yes I'll do it as a separate plugin first Then we can think about including it in the core On Jul 29, 2014 9:46 PM, "Bruno Oliveira" <nicoddemus@gmail.com> wrote:
Hi,
Perhaps this could a plugin (pytest-datadiff)? I know you can customize assertion messages for other types (see http://pytest.org/latest/assert.html#defining-your-own-assertion-comparison), but I don't know how that plays with assertions already built in pytest.
Cheers,
On Tue, Jul 29, 2014 at 5:24 AM, Anatoly Bubenkov <bubenkoff@gmail.com> wrote:
what do you think about extending the existing assert representation with this https://pypi.python.org/pypi/datadiff would be nice if we could extend datadiff with what pytest does for string diff and then just use datadiff for all assertions because for now, it's not too useful to see a comparison of 2 dicts in the pytest it's not readable especially if 2 dicts are big and nowadays, json so dicts are everywhere datadiff is cool at showing the diff between 2 dicts the reason why im asking without the PR is the politics about the dependencies as i know you're not so positive about adding them also this standalone mode is a bit tricky in that case, i guess example:
def test_ticket_filter_on_order_uid__in(app, first_ticket, second_ticket, second_basket_ticket, order, second_order): """Test if the right result is returned when filtering on basket.""" ticket_filter = pta.ticket.filter(api_key=app.uid, order_uid__in=[order['uid']]).content
assert ticket_filter == [first_ticket, second_ticket] E assert [{'code': '91...dulum'}, ...}] == [{'basket_uid'... 'EUR'}, ...}] E At index 0 diff: {'code': '911338060856', 'subtitle': {'en': u'Subtitle1 Kaleidoscope'}, 'uid': u'37bc3c15e66a45e9bf3b52233fcdff02', 'order_uid': u'08dec1bd7ec249109729de448ffcc571', 'price': {'currency': 'EUR', 'amount': '1.20'}, 'fees': [{'price': {'currency': 'EUR', 'amount': '2.00'}, 'type': 'service costs'}], 'product_uid': u'32c2a6f98f0642b9843a8dcce6dead56', 'name': {'en': u'Ticket1 \u2603 Weapon'}} != {'subtitle': {'en': u'Subtitle1 Kaleidoscope'}, 'uid': u'37bc3c15e66a45e9bf3b52233fcdff02', 'price': {'currency': 'EUR', 'amount': '1.20'}, 'fees': [{'price': {'currency': 'EUR', 'amount': '2.00'}, 'type': 'service costs'}], 'basket_uid': u'871e829da04e4cd9a45d244f55824c83', 'product_uid': u'32c2a6f98f0642b9843a8dcce6dead56', 'name': {'en': u'Ticket1 \u2603 Weapon'}} just unreadable
-- Anatoly Bubenkov
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
Nice, be sure to keep the mailing list informed. :) Cheers, On Tue, Jul 29, 2014 at 5:17 PM, Anatoly Bubenkov <bubenkoff@gmail.com> wrote:
Yes I'll do it as a separate plugin first Then we can think about including it in the core On Jul 29, 2014 9:46 PM, "Bruno Oliveira" <nicoddemus@gmail.com> wrote:
Hi,
Perhaps this could a plugin (pytest-datadiff)? I know you can customize assertion messages for other types (see http://pytest.org/latest/assert.html#defining-your-own-assertion-comparison), but I don't know how that plays with assertions already built in pytest.
Cheers,
On Tue, Jul 29, 2014 at 5:24 AM, Anatoly Bubenkov <bubenkoff@gmail.com> wrote:
what do you think about extending the existing assert representation with this https://pypi.python.org/pypi/datadiff would be nice if we could extend datadiff with what pytest does for string diff and then just use datadiff for all assertions because for now, it's not too useful to see a comparison of 2 dicts in the pytest it's not readable especially if 2 dicts are big and nowadays, json so dicts are everywhere datadiff is cool at showing the diff between 2 dicts the reason why im asking without the PR is the politics about the dependencies as i know you're not so positive about adding them also this standalone mode is a bit tricky in that case, i guess example:
def test_ticket_filter_on_order_uid__in(app, first_ticket, second_ticket, second_basket_ticket, order, second_order): """Test if the right result is returned when filtering on basket.""" ticket_filter = pta.ticket.filter(api_key=app.uid, order_uid__in=[order['uid']]).content
assert ticket_filter == [first_ticket, second_ticket] E assert [{'code': '91...dulum'}, ...}] == [{'basket_uid'... 'EUR'}, ...}] E At index 0 diff: {'code': '911338060856', 'subtitle': {'en': u'Subtitle1 Kaleidoscope'}, 'uid': u'37bc3c15e66a45e9bf3b52233fcdff02', 'order_uid': u'08dec1bd7ec249109729de448ffcc571', 'price': {'currency': 'EUR', 'amount': '1.20'}, 'fees': [{'price': {'currency': 'EUR', 'amount': '2.00'}, 'type': 'service costs'}], 'product_uid': u'32c2a6f98f0642b9843a8dcce6dead56', 'name': {'en': u'Ticket1 \u2603 Weapon'}} != {'subtitle': {'en': u'Subtitle1 Kaleidoscope'}, 'uid': u'37bc3c15e66a45e9bf3b52233fcdff02', 'price': {'currency': 'EUR', 'amount': '1.20'}, 'fees': [{'price': {'currency': 'EUR', 'amount': '2.00'}, 'type': 'service costs'}], 'basket_uid': u'871e829da04e4cd9a45d244f55824c83', 'product_uid': u'32c2a6f98f0642b9843a8dcce6dead56', 'name': {'en': u'Ticket1 \u2603 Weapon'}} just unreadable
-- Anatoly Bubenkov
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
Hi, On 29 July 2014 09:24, Anatoly Bubenkov <bubenkoff@gmail.com> wrote:
what do you think about extending the existing assert representation with this https://pypi.python.org/pypi/datadiff would be nice if we could extend datadiff with what pytest does for string diff
So datadiff is not as good as comparing strings then what py.test does now?
and then just use datadiff for all assertions because for now, it's not too useful to see a comparison of 2 dicts in the pytest it's not readable especially if 2 dicts are big
We should always strive to make the assertions better. If you think they can be improved using datadiff then let's improve it. The current reprs where simply derived from unittest2 with some improvements from feedback later.
the reason why im asking without the PR is the politics about the dependencies as i know you're not so positive about adding them
I guess new external dependencies are not ideal. But looking at datadiff it seems to be only about 300 lines of Apache licensed code. That's not a lot of code by any means. So I think there are a few options which I think are realistic: * Vendor datadiff, I think the licenses are compatible. * Re-write what datadiff produces directly inside _pytest/assertion/utils.py. It's not that a crazy task. This is only a bad idea if datadiff keeps evolving a lot, but given it's last commit was in 2012 I think we're fine. * Have datadiff as an optional dependency, if it's importable we use it for comparing stuff, otherwise use py.test's built in code. I guess I'd be -0 on this one. There's also the option of an external plugin which was already mentioned, but I think we should aim for the best exceptions out of the box (which is why i'm -0 on the last option). So I'm all for improving out of the box assertion reprs. The simplest option is vendoring datadiff (maybe with a few tweaks), any objections to that approach? The only downside I see is that the license becomes more complicated. Regards, Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org
On 1 August 2014 23:09, Floris Bruynooghe <flub@devork.be> wrote:
So datadiff is not as good as comparing strings then what py.test does now?
it doesn't even support such comparison - raises notimplemented
We should always strive to make the assertions better. If you think they can be improved using datadiff then let's improve it. The current reprs where simply derived from unittest2 with some improvements from feedback later.
+1
the reason why im asking without the PR is the politics about the dependencies as i know you're not so positive about adding them
I guess new external dependencies are not ideal. But looking at datadiff it seems to be only about 300 lines of Apache licensed code. That's not a lot of code by any means. So I think there are a few options which I think are realistic:
* Vendor datadiff, I think the licenses are compatible.
by vendoring you mean using it as a dependency? if so, then i also prefer this way the author seems doing something at sourgeforge, so he's alive, and we can ask for PR if they will be needed about having datadiff diffs as external plugin, not putting it in the core - that was more like sandboxing approach, i think and i think it's good intention -- Anatoly Bubenkov
On 2 August 2014 10:08, Anatoly Bubenkov <bubenkoff@gmail.com> wrote:
On 1 August 2014 23:09, Floris Bruynooghe <flub@devork.be> wrote:
* Vendor datadiff, I think the licenses are compatible.
by vendoring you mean using it as a dependency?
No, vendoring means taking the datadiff source and including it in the py.test source. So we copy-paste the code and don't have an external dependency. -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org
well, this will also work for me especially knowing that we'll have to extend datadiff anyway to include string comparison On 2 August 2014 13:19, Floris Bruynooghe <flub@devork.be> wrote:
On 2 August 2014 10:08, Anatoly Bubenkov <bubenkoff@gmail.com> wrote:
On 1 August 2014 23:09, Floris Bruynooghe <flub@devork.be> wrote:
* Vendor datadiff, I think the licenses are compatible.
by vendoring you mean using it as a dependency?
No, vendoring means taking the datadiff source and including it in the py.test source. So we copy-paste the code and don't have an external dependency.
-- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org
-- Anatoly Bubenkov
Hi Floris, On Fri, Aug 01, 2014 at 22:09 +0100, Floris Bruynooghe wrote:
Hi,
On 29 July 2014 09:24, Anatoly Bubenkov <bubenkoff@gmail.com> wrote:
what do you think about extending the existing assert representation with this https://pypi.python.org/pypi/datadiff would be nice if we could extend datadiff with what pytest does for string diff
So datadiff is not as good as comparing strings then what py.test does now?
and then just use datadiff for all assertions because for now, it's not too useful to see a comparison of 2 dicts in the pytest it's not readable especially if 2 dicts are big
We should always strive to make the assertions better. If you think they can be improved using datadiff then let's improve it. The current reprs where simply derived from unittest2 with some improvements from feedback later.
the reason why im asking without the PR is the politics about the dependencies as i know you're not so positive about adding them
I guess new external dependencies are not ideal. But looking at datadiff it seems to be only about 300 lines of Apache licensed code. That's not a lot of code by any means. So I think there are a few options which I think are realistic:
* Vendor datadiff, I think the licenses are compatible.
* Re-write what datadiff produces directly inside _pytest/assertion/utils.py. It's not that a crazy task. This is only a bad idea if datadiff keeps evolving a lot, but given it's last commit was in 2012 I think we're fine.
That's the best option IMO.
* Have datadiff as an optional dependency, if it's importable we use it for comparing stuff, otherwise use py.test's built in code. I guess I'd be -0 on this one.
There's also the option of an external plugin which was already mentioned, but I think we should aim for the best exceptions out of the box (which is why i'm -0 on the last option).
So I'm all for improving out of the box assertion reprs. The simplest option is vendoring datadiff (maybe with a few tweaks), any objections to that approach? The only downside I see is that the license becomes more complicated.
Isn't it maybe enough to look at the improved output and implement it? I'd ideally like to avoid licensing complications because i somewhat often get questions from companies and distributors wrt to licensing. But i agree it's not a super-big deal in this case likely. best, holger
Regards, Floris
-- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org _______________________________________________ Pytest-dev mailing list Pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
On 4 August 2014 08:35, holger krekel <holger@merlinux.eu> wrote:
On Fri, Aug 01, 2014 at 22:09 +0100, Floris Bruynooghe wrote:
* Re-write what datadiff produces directly inside _pytest/assertion/utils.py. It's not that a crazy task. This is only a bad idea if datadiff keeps evolving a lot, but given it's last commit was in 2012 I think we're fine.
That's the best option IMO.
Ok, I'm happy with that too (in fact with my conservative hat on this is my preferred option), it will just take a bit longer I suspect.
So I'm all for improving out of the box assertion reprs. The simplest option is vendoring datadiff (maybe with a few tweaks), any objections to that approach? The only downside I see is that the license becomes more complicated.
Isn't it maybe enough to look at the improved output and implement it?
I'd ideally like to avoid licensing complications because i somewhat often get questions from companies and distributors wrt to licensing. But i agree it's not a super-big deal in this case likely.
Yes, keeping the license simple is a valid concern. -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org
participants (4)
-
Anatoly Bubenkov -
Bruno Oliveira -
Floris Bruynooghe -
holger krekel