I'm not 100% about the important differences, so this is a bit of an intuitive analysis from my side (I know little about Hypothesis and more about fuzzing). Hypothesis has support for traditional fuzzing [sic]: https://hypothesis.readthedocs.io/en/latest/details.html?highlight=fuzz#use-... and OSS-Fuzz supports using Python fuzzing by way of Hypothesis https://google.github.io/oss-fuzz/getting-started/new-project-guide/python-l... although it will be seeded with the Atheris fuzzer and based on this issue https://github.com/google/atheris/issues/20 it seems Atheris + Hypothesis might not be working particularly well together. I think based on the above and skimming through the Hypothesis docs that there are many similarities between fuzzing (Atheris specifically) but the underlying engine that explores the input space is different. Fuzzing is coverage-guided (which I don't think Hypothesis is, but I could be wrong), meaning the target program is instrumented to identify if a newly generated input explores new code. In essence, this makes fuzzing a mutational genetic algorithm. Another benefit is OSS-Fuzz will build the target code with various sanitizers (ASan, UBSan, MSan) which will help highlight issues in the native code. About the why it would be great to fuzz more Python code, then this was more of a general statement in that a lot of effort is being put into this from the OSS-Fuzz side because Python is a widely used language. For example, an effort in this domain is investigation into new bug oracles for Python (like sanitizers but targeted memory safe languages). On 07/06/2022 15:10, Matti Picus wrote:
On 7/6/22 14:02, david korczynski wrote:
Hi Numpy maintainers,
Would you be interested in integrating continuous fuzzing by way of OSS-Fuzz? Fuzzing is a way to automate test-case generation and has been heavily used for memory unsafe languages. Recently efforts have been put into fuzzing memory safe languages and Python is one of the languages where it would be great to use fuzzing.
...
Let me know your thoughts on this and if you have any questions as I’m happy to clarify or go more into details with fuzzing.
Kind regards, David
Could you compare and contrast this to hypothesis [0], which we are already using in our testing?
I don't understand what you mean by "Python is one of the languages where it would be great to use fuzzing". Why?
Matti
[0] https://hypothesis.readthedocs.io/en/latest/index.html
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: david@adalogics.com ADA Logics Ltd is registered in England. No: 11624074. Registered office: 266 Banbury Road, Post Box 292, OX2 7DL, Oxford, Oxfordshire , United Kingdom
ADA Logics Ltd is registered in England. No: 11624074. Registered office: 266 Banbury Road, Post Box 292, OX2 7DL, Oxford, Oxfordshire , United Kingdom