[SciPy-User] Announcing Theano 1.0.0

Steven Bocco stevenbocco at gmail.com
Thu Nov 16 12:06:37 EST 2017


 Announcing Theano 1.0.0

This is a release for a major version, with lots of new features, bug
fixes, and some interface changes (deprecated or potentially misleading
features were removed).

Upgrading to Theano 1.0.0 is recommended for everyone, but you should first
make sure that your code does not raise deprecation warnings with Theano
0.9*. Otherwise either results can change, or warnings may have been turned
into errors.

For those using the bleeding edge version in the git repository, we
encourage you to update to the rel-1.0.0 tag.
What's New Highlights (since 0.9.0):

   - Announcing that MILA will stop developing Theano
   <https://groups.google.com/d/msg/theano-users/7Poq8BZutbY/rNCIfvAEAwAJ>
   - conda packages now available and updated in our own conda channel
   mila-udem To install it: conda install -c mila-udem theano pygpu
   - Support NumPy 1.13
   - Support pygpu 0.7
   - Moved Python 3.* minimum supported version from 3.3 to 3.4
   - Added conda recipe
   - Replaced deprecated package nose-parameterized with up-to-date package
   parameterized for Theano requirements
   - Theano now internally uses sha256 instead of md5 to work on systems
   that forbid md5 for security reason
   - Removed old GPU backend theano.sandbox.cuda. New backend
   theano.gpuarray is now the official GPU backend
   - Make sure MKL uses GNU OpenMP
      - *NB*: Matrix dot product (gemm) with mkl from conda could return
      wrong results in some cases. We have reported the problem upstream and we
      have a work around that raises an error with information about how to fix
      it.
   - Improved elemwise operations
      - Speed-up elemwise ops based on SciPy
      - Fixed memory leaks related to elemwise ops on GPU
   - Scan improvements
      - Speed up Theano scan compilation and gradient computation
      - Added meaningful message when missing inputs to scan
   - Speed up graph toposort algorithm
   - Faster C compilation by massively using a new interface for op params
   - Faster optimization step, with new optional destroy handler
   - Documentation updated and more complete
      - Added documentation for RNNBlock
      - Updated conv documentation
   - Support more debuggers for PdbBreakpoint
   - Many bug fixes, crash fixes and warning improvements

Interface changes:

   - Merged duplicated diagonal functions into two ops: ExtractDiag
   (extract a diagonal to a vector), and AllocDiag (set a vector as a
   diagonal of an empty array)
   - Removed op ExtractDiag from theano.tensor.nlinalg, now only in
   theano.tensor.basic
   - Generalized AllocDiag for any non-scalar input
   - Added new parameter target for MRG functions
   - Renamed MultinomialWOReplacementFromUniform to ChoiceFromUniform
   - Changed grad() method to L_op() in ops that need the outputs to
   compute gradient
   - Removed or deprecated Theano flags:
      - cublas.lib
      - cuda.enabled
      - enable_initial_driver_test
      - gpuarray.sync
      - home
      - lib.cnmem
      - nvcc.* flags
      - pycuda.init

Convolution updates:

   - Implemented separable convolutions for 2D and 3D
   - Implemented grouped convolutions for 2D and 3D
   - Added dilated causal convolutions for 2D
   - Added unshared convolutions
   - Implemented fractional bilinear upsampling
   - Removed old conv3d interface
   - Deprecated old conv2d interface

GPU:

   - Added a meta-optimizer to select the fastest GPU implementations for
   convolutions
   - Prevent GPU initialization when not required
   - Added disk caching option for kernels
   - Added method my_theano_function.sync_shared() to help synchronize GPU
   Theano functions
   - Added useful stats for GPU in profile mode
   - Added Cholesky op based on cusolver backend
   - Added GPU ops based on magma library
   <http://icl.cs.utk.edu/magma/software/>: SVD, matrix inverse, QR,
   cholesky and eigh
   - Added GpuCublasTriangularSolve
   - Added atomic addition and exchange for long long values in
   GpuAdvancedIncSubtensor1_dev20
   - Support log gamma function for all non-complex types
   - Support GPU SoftMax in both OpenCL and CUDA
   - Support offset parameter k for GpuEye
   - CrossentropyCategorical1Hot and its gradient are now lifted to GPU
   - cuDNN:
      - Official support for v6.* and v7.*
      - Added spatial transformation operation based on cuDNN
      - Updated and improved caching system for runtime-chosen cuDNN
      convolution algorithms
      - Support cuDNN v7 tensor core operations for convolutions with
      runtime timed algorithms
      - Better support and loading on Windows and Mac
      - Support cuDNN v6 dilated convolutions
      - Support cuDNN v6 reductions for contiguous inputs
      - Optimized SUM(x^2), SUM(ABS(X)) and MAX(ABS(X)) operations with
      cuDNN reductions
      - Added new Theano flags cuda.include_path, dnn.base_path and
      dnn.bin_path to help configure Theano when CUDA and cuDNN can not be
      found automatically
      - Extended Theano flag dnn.enabled with new option no_check to help
      speed up cuDNN importation
      - Disallowed float16 precision for convolution gradients
      - Fixed memory alignment detection
      - Added profiling in C debug mode (with theano flag cmodule.debug=True
      )
      - Added Python scripts to help test cuDNN convolutions
      - Automatic addition of cuDNN DLL path to PATH environment variable
      on Windows
   - Updated float16 support
      - Added documentation for GPU float16 ops
      - Support float16 for GpuGemmBatch
      - Started to use float32 precision for computations that don't
      support float16 on GPU

New features:

   - Implemented truncated normal distribution with box-muller transform
   - Added L_op() overriding option for OpFromGraph
   - Added NumPy C-API based fallback implementation for [sd]gemv_ and
   [sd]dot_
   - Implemented topk and argtopk on CPU and GPU
   - Implemented max() and min() functions for booleans and unsigned
   integers types
   - Added tensor6() and tensor7() in theano.tensor module
   - Added boolean indexing for sub-tensors
   - Added covariance matrix function theano.tensor.cov
   - Added a wrapper for Baidu's CTC
   <https://github.com/baidu-research/warp-ctc> cost and gradient functions
   - Added scalar and elemwise CPU ops for modified Bessel function of
   order 0 and 1 from scipy.special
   - Added Scaled Exponential Linear Unit (SELU) activation
   - Added sigmoid_binary_crossentropy function
   - Added tri-gamma function
   - Added unravel_index and ravel_multi_index functions on CPU
   - Added modes half and full for Images2Neibs ops
   - Implemented gradient for AbstractBatchNormTrainGrad
   - Implemented gradient for matrix pseudoinverse op
   - Added new prop replace for ChoiceFromUniform op
   - Added new prop on_error for CPU Cholesky op
   - Added new Theano flag deterministic to help control how Theano
   optimize certain ops that have deterministic versions. Currently used for
   subtensor Ops only.
   - Added new Theano flag cycle_detection to speed-up optimization step by
   reducing time spending in inplace optimizations
   - Added new Theano flag check_stack_trace to help check the stack trace
   during optimization process
   - Added new Theano flag cmodule.debug to allow a debug mode for Theano C
   code. Currently used for cuDNN convolutions only.
   - Added new Theano flag pickle_test_value to help disable pickling test
   values

Others:

   - Kept stack trace for optimizations in new GPU backend
   - Added deprecation warning for the softmax and logsoftmax vector case
   - Added a warning to announce that C++ compiler will become mandatory in
   next Theano release 0.11
   - Added R_op() for ZeroGrad
   - Added description for rnnblock

Other more detailed changes:

   - Fixed invalid casts and index overflows in theano.tensor.signal.pool
   - Fixed gradient error for elemwise minimum and maximum when compared
   values are the same
   - Fixed gradient for ARange
   - Removed ViewOp subclass during optimization
   - Removed useless warning when profile is manually disabled
   - Added tests for abstract conv
   - Added options for disconnected_outputs to Rop
   - Removed theano/compat/six.py
   - Removed COp.get_op_params()
   - Support of list of strings for Op.c_support_code(), to help not
   duplicate support codes
   - Macro names provided for array properties are now standardized in both
   CPU and GPU C codes
   - Moved all C code files into separate folder c_code in every Theano
   module
   - Many improvements for Travis CI tests (with better splitting for
   faster testing)
   - Many improvements for Jenkins CI tests: daily testings on Mac and
   Windows in addition to Linux

Download and Install

You can download Theano from http://pypi.python.org/pypi/Theano

Installation instructions are available at
http://deeplearning.net/software/theano/install.html
Description

Theano is a Python library that allows you to define, optimize, and
efficiently evaluate mathematical expressions involving multi-dimensional
arrays. It is built on top of NumPy. Theano features:


   - tight integration with NumPy: a similar interface to NumPy's.
   numpy.ndarrays are also used internally in Theano-compiled functions.
   - transparent use of a GPU: perform data-intensive computations much
   faster than on a CPU.
   - efficient symbolic differentiation: Theano can compute derivatives for
   functions of one or many inputs.
   - speed and stability optimizations: avoid nasty bugs when computing
   expressions such as log(1+ exp(x)) for large values of x.
   - dynamic C code generation: evaluate expressions faster.
   - extensive unit-testing and self-verification: includes tools for
   detecting and diagnosing bugs and/or potential problems.

Theano has been powering large-scale computationally intensive scientific
research since 2007, but it is also approachable enough to be used in the
classroom (IFT6266 at the University of Montreal).
Resources

About Theano:

http://deeplearning.net/software/theano/

Theano-related projects:

http://github.com/Theano/Theano/wiki/Related-projects

About NumPy:

http://numpy.scipy.org/

About SciPy:

http://www.scipy.org/

Machine Learning Tutorial with Theano on Deep Architectures:

http://deeplearning.net/tutorial/
Acknowledgments

I would like to thank all contributors of Theano. Since release 0.9.0, many
people have helped, notably (in alphabetical order):


   - Aarni Koskela
   - Adam Becker
   - Adam Geitgey
   - Adrian Keet
   - Adrian Seyboldt
   - Aleksandar Botev
   - Alexander Matyasko
   - amrithasuresh
   - Andrei Costinescu
   - Anirudh Goyal
   - Anmol Sahoo
   - Arnaud Bergeron
   - Bogdan Budescu
   - Boris Fomitchev
   - Cesar Laurent
   - Chiheb Trabelsi
   - Chong Wu
   - dareneiri
   - Daren Eiri
   - Dzmitry Bahdanau
   - erakra
   - Faruk Ahmed
   - Florian Bordes
   - fo40225
   - Frederic Bastien
   - Gabe Schwartz
   - Ghislain Antony Vaillant
   - Gijs van Tulder
   - Holger Kohr
   - Jan Schlüter
   - Jayanth Koushik
   - Jeff Donahue
   - jhelie
   - João Victor Tozatti Risso
   - Joseph Paul Cohen
   - Juan Camilo Gamboa Higuera
   - Laurent Dinh
   - Lilian Besson
   - lrast
   - Lv Tao
   - Matt Graham
   - Michael Manukyan
   - Mohamed Ishmael Diwan Belghazi
   - Mohammed Affan
   - morrme
   - mrTsjolder
   - Murugesh Marvel
   - naitonium
   - NALEPA
   - Nan Jiang
   - Pascal Lamblin
   - Ramana Subramanyam
   - Rebecca N. Palmer
   - Reyhane Askari
   - Saizheng Zhang
   - Shawn Tan
   - Shubh Vachher
   - Simon Lefrancois
   - Sina Honari
   - Steven Bocco
   - Tegan Maharaj
   - Thomas George
   - Tim Cooijmans
   - Vikram
   - vipulraheja
   - wyjw
   - Xavier Bouthillier
   - xiaoqie
   - Yikang Shen
   - Zhouhan LIN
   - Zotov Yuriy

Also, thank you to all NumPy and Scipy developers as Theano builds on their
strengths.

All questions/comments are always welcome on the Theano mailing-lists (
http://deeplearning.net/software/theano/#community )


-- 
Steven Bocco
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20171116/63003cbf/attachment-0001.html>


More information about the SciPy-User mailing list