Mailman 3 - POT - python.org

Question about ot. gromov_wasserstein function
by Azim Dehghani Amirabad 15 Jul '21

15 Jul '21

Hi, I'm using OT package for GW, and I have a question regarding two arguments of the ot.gromov.gromov_wasserstein function. What is p and q? I couldn't see them in the objective function of the GW. p = ot.unif(n_samples)q = ot.unif(n_samples) gw0, log0 = ot.gromov.gromov_wasserstein( C1, C2, p, q, 'square_loss', verbose=True, log=True) gw, log = ot.gromov.entropic_gromov_wasserstein( C1, C2, p, q, 'square_loss', epsilon=5e-4, log=True, verbose=True) print('Gromov-Wasserstein distances: ' + str(log0['gw_dist']))print('Entropic Gromov-Wasserstein distances: ' + str(log['gw_dist']))

3 2

POT 0.7.0 Released
by Rémi Flamary 07 Jul '21

07 Jul '21

It is with great pleasure that in announce on the list the release of POT 0.7.0 (available also on PyPI and conda-forge). This is the new stable release for POT. We made a lot of changes in the documentation and added several new features such as Partial OT, Unbalanced and Multi Sources OT Domain Adaptation and several bug fixes. One important change is that we have created the GitHub organization PythonOT that now owns the main POT repository https://github.com/PythonOT/POT and the repository for the new documentation is now hosted at https://PythonOT.github.io/. This is the first release where the Python 2.7 tests have been removed. Most of the toolbox should still work but we do not offer support for Python 2.7 and will close related Issues. A lot of changes have been done to the documentation that is now hosted on https://PythonOT.github.io/ instead of readthedocs. It was a hard choice but readthedocs did not allow us to run sphinx-gallery to update our beautiful examples and it was a huge amount of work to maintain. The documentation is now automatically compiled and updated on merge. We also removed the notebooks from the repository for space reason and also because they are all available in the example gallery. Note that now the output of the documentation build for each commit in the PR is available to check that the doc builds correctly before merging which was not possible with readthedocs. The CI framework has also been changed with a move from Travis to Github Action which allows to get faster tests on Windows, MacOS and Linux. We also now report our coverage on Codecov.io and we have a reasonable 92% coverage. We also now generate wheels for a number of OS and Python versions at each merge in the master branch. They are available as outputs of this action. This will allow simpler multi-platform releases from now on. In terms of new features we now have OTDA Classes for unbalanced OT, a new Domain adaptation class form multi domain problems (JCPOT), and several solvers to solve the Partial Optimal Transport problems. This release is also the moment to thank all the POT contributors (old and new) for helping making POT such a nice toolbox. A lot of changes (also in the API) are comming for the next versions. New Features - New documentation on https://PythonOT.github.io/ (PR #160, PR #143, PR #144) - Documentation build on CircleCI with sphinx-gallery (PR #145,PR #146, #155) - Run sphinx gallery in CI (PR #146) - Remove notebooks from repo because available in doc (PR #156) - Build wheels in CI (#157) - Move from travis to GitHub Action for Windows, MacOS and Linux (PR #148, PR #150) - Partial Optimal Transport (PR#141 and PR #142) - Laplace regularized OTDA (PR #140) - Multi source DA with target shift (PR #137) - Screenkhorn algorithm (PR #121) Closed issues - Bug in Unbalanced OT example (Issue #127) - Clean Cython output when calling setup.py clean (Issue #122) - Various Macosx compilation problems (Issue #113, Issue #118, PR#130) - EMD dimension mismatch (Issue #114, Fixed in PR #116) - 2D barycenter bug for non square images (Issue #124, fixed in PR #132) - Bad value in EMD 1D (Issue #138, fixed in PR #139) - Log bugs for Gromov-Wassertein solver (Issue #107, fixed in PR #108) - Weight issues in barycenter function (PR #106) -- Rémi Flamary Web: http://remi.flamary.com Tel: +33 (0)4 92 07 63 80 Laboratoire Lagrange, UMR CNRS 7293 Observatoire de la Côte d'Azur Université de Nice Sophia-Antipolis

2 1

joining the POT slack channel discussion if you don't have a @unice.fr email address
by Robin Lougee 30 Dec '19

30 Dec '19

I don't have an @unice.fr email address. How do I join the discussion on the POT slack channel? ---------------------------------------------------------------------------------- Robin Lougee, PhD Mobile, Solutions, and Mathematical Sciences - IBM Research Research Industry Lead - Consumer Products / Agriculture IBM Industry Academy Member IBM TJ Watson Research Center 1101 Kitchawan Road, Yorktown Heights, NY 10598 ph: 914-945-3032 fax: 914-945-3434 rlougee(a)us.ibm.com Program Manager, COIN-OR http://www.coin-or.org

3 3

Error in Barycentre computation
by fpm16shikhav＠iima.ac.in 23 Sep '19

23 Sep '19

Dear all I followed the 1D Wasserstein Barycentre demo on Github https://github.com/rflamary/POT/blob/master/notebooks/plot_barycenter_1D.ip… and extended it to my use case where I have 100 pdfs/distributions with 5 bins each. I assign equal weights to each distribution and regularisation parameter value is same as the demo. So the dimensions of dataframes in my code are: A (5,100) M (5,5) weights (100) But on executing my code, I get the following error: C:\Users\Shikha\AppData\Roaming\Python\Python36\site-packages\ot\bregman.py:1049: RuntimeWarning: overflow encountered in true_divide u = (u.T * geometricBar(weights, UKv)).T / UKv Seeking help in resolving this error. Best, Shikha

2 1

Objective for choosing optimal value for epsilon in GW
by Azim Dehghani Amirabad 12 Aug '19

12 Aug '19

Hi, I'm using OT package for GW, and I have a question regarding choosing the optimal value for entropic regularization. What is the good objective to choose epsilon and what are the ranges to try. gw, log = ot.gromov.entropic_gromov_wasserstein( C1, C2, p, q, 'square_loss', epsilon=5e-4, log=True, verbose=True) Best, Azim

1 0

problem instaling POT on mac using pip for python3
by Olivier de Viron 12 Jul '19

12 Jul '19

Hello, I cannot install PO on my using either pip or setup. I got the following message, after entering: pip install POT ERROR: Complete output from command /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 -u -c 'import setuptools, tokenize;__file__='"'"'/private/var/folders/73/xkvgyzh906nc52ydlh23cnvm0000gp/T/pip-install-nzpvw23r/POT/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/73/xkvgyzh906nc52ydlh23cnvm0000gp/T/pip-record-o4wa2u7m/install-record.txt --single-version-externally-managed --compile: ERROR: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type' warnings.warn(msg) running install running build running build_py creating build creating build/lib.macosx-10.6-intel-3.6 creating build/lib.macosx-10.6-intel-3.6/ot copying ot/bregman.py -> build/lib.macosx-10.6-intel-3.6/ot copying ot/plot.py -> build/lib.macosx-10.6-intel-3.6/ot copying ot/stochastic.py -> build/lib.macosx-10.6-intel-3.6/ot copying ot/datasets.py -> build/lib.macosx-10.6-intel-3.6/ot copying ot/__init__.py -> build/lib.macosx-10.6-intel-3.6/ot copying ot/dr.py -> build/lib.macosx-10.6-intel-3.6/ot copying ot/utils.py -> build/lib.macosx-10.6-intel-3.6/ot copying ot/smooth.py -> build/lib.macosx-10.6-intel-3.6/ot copying ot/gromov.py -> build/lib.macosx-10.6-intel-3.6/ot copying ot/optim.py -> build/lib.macosx-10.6-intel-3.6/ot copying ot/da.py -> build/lib.macosx-10.6-intel-3.6/ot creating build/lib.macosx-10.6-intel-3.6/ot/gpu copying ot/gpu/bregman.py -> build/lib.macosx-10.6-intel-3.6/ot/gpu copying ot/gpu/__init__.py -> build/lib.macosx-10.6-intel-3.6/ot/gpu copying ot/gpu/utils.py -> build/lib.macosx-10.6-intel-3.6/ot/gpu copying ot/gpu/da.py -> build/lib.macosx-10.6-intel-3.6/ot/gpu creating build/lib.macosx-10.6-intel-3.6/ot/lp copying ot/lp/__init__.py -> build/lib.macosx-10.6-intel-3.6/ot/lp copying ot/lp/cvx.py -> build/lib.macosx-10.6-intel-3.6/ot/lp creating build/lib.macosx-10.6-intel-3.6/ot/externals copying ot/externals/funcsigs.py -> build/lib.macosx-10.6-intel-3.6/ot/externals copying ot/externals/__init__.py -> build/lib.macosx-10.6-intel-3.6/ot/externals running build_ext building 'ot.lp.emd_wrap' extension creating build/temp.macosx-10.6-intel-3.6 creating build/temp.macosx-10.6-intel-3.6/ot creating build/temp.macosx-10.6-intel-3.6/ot/lp /usr/bin/clang -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I//usr/local/Cellar/gcc/7.3.0/include/c++/7.3.0 -Iot/lp -I/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/include -I/private/var/folders/73/xkvgyzh906nc52ydlh23cnvm0000gp/T/pip-install-nzpvw23r/POT/ot/lp -I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c ot/lp/emd_wrap.cpp -o build/temp.macosx-10.6-intel-3.6/ot/lp/emd_wrap.o warning: include path for stdlibc++ headers not found; pass '-std=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found] In file included from ot/lp/emd_wrap.cpp:28: In file included from /Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m/Python.h:34: In file included from //usr/local/Cellar/gcc/7.3.0/include/c++/7.3.0/stdlib.h:36: In file included from //usr/local/Cellar/gcc/7.3.0/include/c++/7.3.0/cstdlib:77: //usr/local/Cellar/gcc/7.3.0/include/c++/7.3.0/bits/std_abs.h:102:7: error: __float128 is not supported on this target abs(__float128 __x) ^ //usr/local/Cellar/gcc/7.3.0/include/c++/7.3.0/bits/std_abs.h:101:3: error: __float128 is not supported on this target __float128 ^ In file included from ot/lp/emd_wrap.cpp:636: In file included from /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4: In file included from /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:12: In file included from /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1822: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings] #warning "Using deprecated NumPy API, disable it with " \ ^ In file included from ot/lp/emd_wrap.cpp:638: In file included from ot/lp/EMD.h:21: ot/lp/network_simplex_simple.h:1421:20: warning: unused variable 'prevCost' [-Wunused-variable] double prevCost=-1; ^ 3 warnings and 2 errors generated. error: command '/usr/bin/clang' failed with exit status 1 ---------------------------------------- ERROR: Command "/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 -u -c 'import setuptools, tokenize;__file__='"'"'/private/var/folders/73/xkvgyzh906nc52ydlh23cnvm0000gp/T/pip-install-nzpvw23r/POT/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/73/xkvgyzh906nc52ydlh23cnvm0000gp/T/pip-record-o4wa2u7m/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/73/xkvgyzh906nc52ydlh23cnvm0000gp/T/pip-install-nzpvw23r/POT/ It occured after I add the path for iostream, which was getting me an error message, using export CPPFLAGS=-I//usr/local/Cellar/gcc/7.3.0/include/c++/7.3.0 I also tried to use python setup install, but the same occurs. I am working on a mac notebook pro with High Sierra OS. Does someone have any idea to help? Thanks! odv -- Olivier de viron Directeur de l'umr lienss DEPARTEMENT SCIENCES DE LA TERRE 05 46 50 76 34 olivier.de_viron(a)univ-lr.fr <mailto:olivier.de_viron@univ-lr.fr> LIttoral ENvironnement et Sociétés (LIENSs) *Université de La Rochelle* Bâtiment ILE 2, rue Olympe de Gouges 17000 La Rochelle lienss.univ-larochelle.fr <http://lienss.univ-larochelle.fr> www.univ-larochelle.fr <https://www.univ-larochelle.fr> Suivez-nous : Facebook <https://www.facebook.com/Universite.de.La.Rochelle> | Twitter <https://twitter.com/UnivLaRochelle>

1 0

Re: Ask for help! POT ot.gpu can't find gpu module.
by Rémi Flamary 05 Apr '19

05 Apr '19

Hello Lu, Thank you for your interest in POT. By default the gpu module is not imported because it depends on cupy that might not be available (its not a hard dependency for POT). So in order to use it you need to import like this: import ot import ot.gpu To be frank the gpu module is not particularly optimized but in order to get the most out of it you should send all your data on the GPU as cupy arrays before using the functions or else there will be some important overhead. Depending on what you want to do reimplementing the very simple Sinkhorn in pytorch might lead to better performances. For large scale computation of Wassertsein distance and gradients I recommend the geomloss toolbox https://www.kernel-operations.io/geomloss/ Is nicely recompute the cost matrix online which makes it very memory efficient and can handle a large number of samples. Sincerly, Rémi Le 04/04/2019 à 20:53, Chen, Lu a écrit : > Dear developers of POT, > > When I use ot.gpu there is a mistake 'module' object has no attribute > 'gpu' and I found that the ot dist has no gpu module. I have tried both > in python 2 and python3 but I haven't solved this problem. Can you help > me? Thank you very much! > > Best regards > > -- > Lu Chen > Department of Informatics > Ying Wu School of Computing > New Jersey Institute of Technology > Newark, NJ 07102-1982 > > > -- Rémi Flamary Web: https://remi.flamary.com Tel: +33 (0)4 92 07 63 80 Laboratoire Lagrange, UMR CNRS 7293 Observatoire de la Côte d'Azur Université Côte d'Azur

1 0

Barycentric Coordinates 2D Interpolation Problem
by Aruther5＠gmu.edu 03 Jul '18

03 Jul '18

Hi, I am currently trying to use the bregman barycenter algorithm to interpolate between 2D floating point arrays. I am getting some odds results and was hoping to better understand what's happening. The example I've attached illustrates the main issue. In this example, I'm trying to interpolate between two 2D arrays, one with 1s in the upper-right quadrant, and 0s elsewhere, the other with 1s in the lower left quadrant, and 0s elsewhere. I'm then using barycenter with different weights ([1-t, t] for t in {0, 0.25, 0.5, 0.75, 1}). What I would have expected is that each output would have the same number of 1s, just in different locations. for t = 0, I'd expect to get my first input, for t = 1, my second input. For t=0.5, one quadrant's-worth of 1s at the center of the output image. Instead, the output I'm getting is fuzzy, and also significantly offset from where I'd expect for all t other than 0.5. At the very least, I can't imaging why I would get anything other than my original input for t = 0 or 1. Do I misunderstand the function, or have I made an error somewhere? If anyone could help me to better understand what's happening here, I would appreciate it. Here the code I'm using: https://pastebin.com/EuW2sAUM And here's the output of the above code: https://imgur.com/a/H9AK9

3 2

Nearest Neighbor Classifier -OTDA
by Kowshik Thopalli 26 Jun '18

26 Jun '18

Hello , I am following these two papers- Optimal Transport for Domain adaptation Courty et al in TPAMI [1] and Large scale OT and Mapping estimation -Seguey et al in ICLR[2] The papers say that they use Nearest Neighbor classifier as the baseline and I am confused about it. As in [1] I took 1800 Usps and 2000 MNIST dataset and using the OTDA examples found the transported source. I am struck after this for the classification i.e., do I now compute a 2000 X 1800 distance matrix between target and source and use nearest neighbors? Here is the code that I am using. Can you please tell me how to proceed from here ##% experiment with domain adaptation import ot import numpy as np import sklearn import matplotlib.pyplot as plt from scipy.io import loadmat #%% data u_m=loadmat('USPS_vs_MNIST.mat') Xs= u_m['X_src'] Ys=u_m['Y_src'] Xt=u_m['X_tar'] Yt=u_m['Y_tar'] #%% DA ot_sinkhorn_un = ot.da.SinkhornTransport(reg_e=1) ot_sinkhorn_un.fit(Xs=Xs.T, Xt=Xt.T) transp_Xs_sinkhorn_un = ot_sinkhorn_un.transform(Xs=Xs.T).T #%% nearest neigbor classifier from sklearn.metrics import confusion_matrix as CM import scipy M=scipy.spatial.distance.cdist(Xt.T,transp_Xs_sinkhorn_un.T,'sqeuclidean')# not correct a=np.argmin(M,axis=1) labels_test= Yt[a] conf= CM(Yt,labels_test) acc= np.sum(np.diag(conf))/np.sum(conf) acc

1 1

Fwd: Understanding 1-D barycenters in 1-D
by Kowshik Thopalli 10 Jun '18

10 Jun '18

Hello, I am using the following modified code from the tutorial examples/plot_barycenter_1D.py. I have just modified it to compute barycenters between bimodal 1-D distributions. I expected to see the barycenter to also contain only two peaks however it contains 3 which is not very intuitive geometrically speaking . I have attached the resultant figure showing that the barycenter computed has three peaks. Is there a way to intuitively understand why this is happening. The results dont show this if weights computed using \alpha when \alpha = {0,0.1,0.8,0.9,1} Thanks Kowshik The code is here - https://pastebin.com/bgMgrBMd The result image is here- https://imgur.com/a/o3ptZ8U <https://imgur.com/a/o3ptZ8U> ᐧ

1 0