From stefan at sun.ac.za  Thu Sep  9 18:52:05 2010
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Thu, 9 Sep 2010 15:52:05 -0700 (PDT)
Subject: contributing C/Cython code to the scikit
In-Reply-To: <20100909214336.GB23543@phare.normalesup.org>
References: <20100909214336.GB23543@phare.normalesup.org>
Message-ID: <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>

Hi Emmanuelle

On Sep 9, 10:43?pm, Emmanuelle Gouillart
<emmanuelle.gouill... at normalesup.org> wrote:
> ? ? ? ? I'm in contact with researchers in image processing and computer
> vision who have developed a C library for image processing. As they are
> also using Python, they may be interested in contributing some parts of
> their existing code to the scikit. I would like to know if it is
> compatible with the development policy of the scikit to include C code
> wrapped with cython in order to use numpy arrays (provided there is a
> significant speed benefit, and that good documentation and tests are
> provided, of course), or if pure Python/Cython is required.

Since we already require a C compiler (by allowing Cython code), I see
no problem with this approach.  All contributions are much
appreciated!  My time to work on this project has been limited
recently, and I would love to get more hands on board.

Regards
St?fan


From sccolbert at gmail.com  Thu Sep  9 19:22:37 2010
From: sccolbert at gmail.com (Chris Colbert)
Date: Thu, 9 Sep 2010 19:22:37 -0400
Subject: contributing C/Cython code to the scikit
In-Reply-To: <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
References: <20100909214336.GB23543@phare.normalesup.org>
	<b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
Message-ID: <AANLkTik-LnV4vmdVm1H3P3zqECq_g+M_npctVgJ55hX4@mail.gmail.com>

+ 1

On Thu, Sep 9, 2010 at 6:52 PM, Stefan van der Walt <stefan at sun.ac.za>wrote:

> Hi Emmanuelle
>
> On Sep 9, 10:43 pm, Emmanuelle Gouillart
> <emmanuelle.gouill... at normalesup.org> wrote:
> >         I'm in contact with researchers in image processing and computer
> > vision who have developed a C library for image processing. As they are
> > also using Python, they may be interested in contributing some parts of
> > their existing code to the scikit. I would like to know if it is
> > compatible with the development policy of the scikit to include C code
> > wrapped with cython in order to use numpy arrays (provided there is a
> > significant speed benefit, and that good documentation and tests are
> > provided, of course), or if pure Python/Cython is required.
>
> Since we already require a C compiler (by allowing Cython code), I see
> no problem with this approach.  All contributions are much
> appreciated!  My time to work on this project has been limited
> recently, and I would love to get more hands on board.
>
> Regards
> St?fan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20100909/8d80a021/attachment.html>

From emmanuelle.gouillart at normalesup.org  Thu Sep  9 17:43:36 2010
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Thu, 9 Sep 2010 23:43:36 +0200
Subject: contributing C/Cython code to the scikit
Message-ID: <20100909214336.GB23543@phare.normalesup.org>

	Hello,

	I'm in contact with researchers in image processing and computer
vision who have developed a C library for image processing. As they are
also using Python, they may be interested in contributing some parts of
their existing code to the scikit. I would like to know if it is
compatible with the development policy of the scikit to include C code
wrapped with cython in order to use numpy arrays (provided there is a
significant speed benefit, and that good documentation and tests are
provided, of course), or if pure Python/Cython is required. 

	I know some of their stuff which is really cool, like top-notch
denoising and segmentation, so I'm planning to give them a hand anyway in
order to cythonize the code, but it would be even better if some of the
work could be contributed to the scikit.


	So tell me about your opinion on C code in the scikit :-D.

	Cheers,

	Emmanuelle


From emmanuelle.gouillart at normalesup.org  Fri Sep 10 08:56:11 2010
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Fri, 10 Sep 2010 14:56:11 +0200
Subject: contributing C/Cython code to the scikit
In-Reply-To: <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
Message-ID: <20100910125611.GA31060@phare.normalesup.org>

	Hi St??????fan and Chris,

	thanks for the constructive answer! I'll keep you informed on the
mailing-list.

	Cheers,

	Emmanuelle

On Thu, Sep 09, 2010 at 03:52:05PM -0700, Stefan van der Walt wrote:
> Hi Emmanuelle

> On Sep 9, 10:43??????pm, Emmanuelle Gouillart
> <emmanuelle.gouill... at normalesup.org> wrote:
> > ?????? ?????? ?????? ?????? I'm in contact with researchers in image processing and computer
> > vision who have developed a C library for image processing. As they are
> > also using Python, they may be interested in contributing some parts of
> > their existing code to the scikit. I would like to know if it is
> > compatible with the development policy of the scikit to include C code
> > wrapped with cython in order to use numpy arrays (provided there is a
> > significant speed benefit, and that good documentation and tests are
> > provided, of course), or if pure Python/Cython is required.

> Since we already require a C compiler (by allowing Cython code), I see
> no problem with this approach.  All contributions are much
> appreciated!  My time to work on this project has been limited
> recently, and I would love to get more hands on board.

> Regards
> St??????fan


From mael.primet at gmail.com  Mon Sep 20 13:02:16 2010
From: mael.primet at gmail.com (mael)
Date: Mon, 20 Sep 2010 10:02:16 -0700 (PDT)
Subject: contributing C/Cython code to the scikit
In-Reply-To: <20100910125611.GA31060@phare.normalesup.org>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org>
Message-ID: <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>

Hello everyone,

here is a first code draft, including two basic filters
 http://github.com/maelp/scikits.image

the goal is not to include high-profile algorithms right now, but to
get the base framework correctly (and then we'll have some engineer
translate our codebase to python).
could you tell me if the way I started it seems okay  with
 you? (eg I've been hacking in some files to have everything compile..
am I doing it as it should be done?)

feel free to comment on code and possible  issues , as we're trying to
have some solid base to build upon :)


From stefan at sun.ac.za  Mon Sep 20 16:37:42 2010
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Mon, 20 Sep 2010 22:37:42 +0200
Subject: contributing C/Cython code to the scikit
In-Reply-To: <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
Message-ID: <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com>

Hi Mael

Welcome!

On Mon, Sep 20, 2010 at 7:02 PM, mael <mael.primet at gmail.com> wrote:
> here is a first code draft, including two basic filters
> ?http://github.com/maelp/scikits.image

First feedback: be sure to update the contributors.txt.  We won't
accept patches without it :)

> could you tell me if the way I started it seems okay ?with
> ?you? (eg I've been hacking in some files to have everything compile..
> am I doing it as it should be done?)

Some general comments:

- We follow PEP7 and 8 regarding code style
- Docstrings with usable examples are really important. We have a
"plot" directive for including those in the docs as well.
- Unit tests should be included in the standard test suite.  All
functions, including utility functions, should be tested and
documented.
- The 'shapes' sub-module is included twice?
- Your patch removes the 'opencv' module for some reason.

At the last sprint, we briefly spoke about how to handle different
data types.  IIRC, the idea was to support float and integer arrays,
and to write a general utility function, used by most other functions,
to convert the input image appropriately.  This isn't in place yet, so
we'll leave that discussion for another day.

Thanks again for contributing!

Regards
St?fan


From mael.primet at gmail.com  Tue Sep 21 05:10:24 2010
From: mael.primet at gmail.com (mael)
Date: Tue, 21 Sep 2010 02:10:24 -0700 (PDT)
Subject: contributing C/Cython code to the scikit
In-Reply-To: <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
 <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com>
Message-ID: <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com>

Thanks for the input St?fan,

I will try to gradually improve my python code style, have you
specifics you have seen in my code that were out of the way?

I removed opencv on my laptop because this wouldn't build (I'll try to
figure out why)

for the data types, the discussion is very interesting,
most of the algorithms could be adapted for different data-types by
some preprocessor-like translation,
however, this might create some problems (floating point comparison
etc). Plus, our feeling with the previous
library we were using (Megawave) was that it was better not to provide
an algorithm for each and every possible datatype,
as it forced the end-user to ponder whether the algorithm he was
trying to use was suited to his case etc.
But I concur that writing float (possibly float and doubles) and
integer (int and unsigned char?) arrays should be
 made as easy as possible.

I'm in the process of writing docstrings and some unit testing

I'll come back to you once I'll have played a bit with the library,

hope we'll gain some momentum by adding a few interesting algorithms
and get people to use the library as the Python standard

btw we're also interested in coding some powerful visualization
function. I guess we'd rather not clutter the scikit with dependencies
to pyQt or other libraries.
How would you go about doing something like this? Use some library
like chaco? Write from scratch a PyQt viewer?

I'm not really that good a coder, so I'd be happy to have you guys
refactor some of my code and tell me the best way to go about it,
particularly to avoid
copying arrays over and over. We want to have simple contiguous arrays
though, because we don't want to bother with complex array iteration
in our C code.
Is the way I'm doing it (that was given to us by Emmanuelle) the
proper way of getting our arrays as contiguous C arrays or are there
some more efficient
and/or  elegant way to do this?




From mael.primet at gmail.com  Tue Sep 21 15:53:39 2010
From: mael.primet at gmail.com (mael)
Date: Tue, 21 Sep 2010 12:53:39 -0700 (PDT)
Subject: contributing C/Cython code to the scikit
In-Reply-To: <20100921192211.GD31763@phare.normalesup.org>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
 <20100921192211.GD31763@phare.normalesup.org>
Message-ID: <8ef1fdde-d31e-493e-adfe-55f2f3b14f42@m16g2000vbs.googlegroups.com>

Thanks Emmanuelle for your code refactoring and the remarks,

> * if there is no good reason to keep lena256.tif (if you can use
> ? scipy.lena() in all your examples and tests), then why not remove the
> ? file.
sure

> I did not have the time to read thoroughly the code of your median
> filter, I think you wanted to know if this was efficient Cython code or
> not. I'll do it soon, maybe after you write a test function for this
> filter? By the way, what if the advantage of this function compared to
> scipy.ndimage.median_filter?
No there is no reason to use our own rather than scipy's one (except
that my implementation might be less obfuscated
because it is less complicated) it was more an example of what we
could do to bind C and python



From stefan at sun.ac.za  Tue Sep 21 11:32:53 2010
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Tue, 21 Sep 2010 17:32:53 +0200
Subject: contributing C/Cython code to the scikit
In-Reply-To: <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
 <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com> <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com>
Message-ID: <AANLkTik33Trt9Q-Tnya0jsqUExP0FB8o4Cy3bKrGKzkK@mail.gmail.com>

Hi Mael

On Tue, Sep 21, 2010 at 11:10 AM, mael <mael.primet at gmail.com> wrote:
> I will try to gradually improve my python code style, have you
> specifics you have seen in my code that were out of the way?

I think mostly spacing issues; but those are well described in the PEP.

> I removed opencv on my laptop because this wouldn't build (I'll try to
> figure out why)

No problem; just make sure that those changes don't get committed to your repo!

> for the data types, the discussion is very interesting,
> most of the algorithms could be adapted for different data-types by
> some preprocessor-like translation,
> however, this might create some problems (floating point comparison
> etc). Plus, our feeling with the previous
> library we were using (Megawave) was that it was better not to provide
> an algorithm for each and every possible datatype,
> as it forced the end-user to ponder whether the algorithm he was
> trying to use was suited to his case etc.
> But I concur that writing float (possibly float and doubles) and
> integer (int and unsigned char?) arrays should be
> ?made as easy as possible.

Our thought was that two types may be manageable, but while we could
always generalise to support more types, it's very hard to go back --
so I'm happy that we're taking a conservative approach here.

> btw we're also interested in coding some powerful visualization
> function. I guess we'd rather not clutter the scikit with dependencies
> to pyQt or other libraries.
> How would you go about doing something like this? Use some library
> like chaco? Write from scratch a PyQt viewer?

You'll be glad to discover that we already have a Qt visualisation
plugin!  Different "imshow" implementations are available, depending
on what is available on your system.  Try this:

import scikits.image.io as sio
print sio.plugins()
sio.use_plugin('qt')
imshow(...)

It should pop up a qt viewer.  You may also do

imshow(..., fancy=True)

for a display with tuning widgets (only supported on Qt currently).

> I'm not really that good a coder, so I'd be happy to have you guys
> refactor some of my code and tell me the best way to go about it,
> particularly to avoid
> copying arrays over and over. We want to have simple contiguous arrays
> though, because we don't want to bother with complex array iteration
> in our C code.

You'll probably find that, especially for large images, it is often
quicker to do a single copy (to get contiguous data) and then perform
your operation.  Good coding comes with practise, and we're all still
learning, so you're in good company.

> Is the way I'm doing it (that was given to us by Emmanuelle) the
> proper way of getting our arrays as contiguous C arrays or are there
> some more efficient
> and/or ?elegant way to do this?

The clearest way is probably `numpy.ascontiguousarray`.

Regards
St?fan


From emmanuelle.gouillart at normalesup.org  Tue Sep 21 12:05:23 2010
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Tue, 21 Sep 2010 18:05:23 +0200
Subject: denoising algorithms / working with Github
In-Reply-To: <AANLkTik33Trt9Q-Tnya0jsqUExP0FB8o4Cy3bKrGKzkK@mail.gmail.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com> <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com> <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com> <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com> <AANLkTik33Trt9Q-Tnya0jsqUExP0FB8o4Cy3bKrGKzkK@mail.gmail.com>
Message-ID: <20100921160523.GB31763@phare.normalesup.org>


Hi Ma??????l,

> On Tue, Sep 21, 2010 at 11:10 AM, mael <mael.primet at gmail.com> wrote:
> > I will try to gradually improve my python code style, have you
> > specifics you have seen in my code that were out of the way?

> I think mostly spacing issues; but those are well described in the PEP.

I'm working on your code right now, so if you're making changes... to
your changes, try to fetch first from my repository. 

Cheers,

Emmanuelle


From sccolbert at gmail.com  Tue Sep 21 18:52:38 2010
From: sccolbert at gmail.com (Chris Colbert)
Date: Tue, 21 Sep 2010 18:52:38 -0400
Subject: contributing C/Cython code to the scikit
In-Reply-To: <20100921195702.GF31763@phare.normalesup.org>
References: <20100909214336.GB23543@phare.normalesup.org>
	<b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
	<20100910125611.GA31060@phare.normalesup.org>
	<c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
	<20100921192211.GD31763@phare.normalesup.org>
	<8ef1fdde-d31e-493e-adfe-55f2f3b14f42@m16g2000vbs.googlegroups.com>
	<20100921195702.GF31763@phare.normalesup.org>
Message-ID: <AANLkTimHWr3sOdusYm3xwFUnc98=ZHxUR6tsuA1atMR=@mail.gmail.com>

So, we definitely are not wanting to re-invent the wheel, unless there is a
clear reason why our wheel is better the previous one.

In short, most of the scikit is focused on the manipulation of 2D images,
whereas scipy.ndimage works on ndimages. So if you can code an equivalent
algorithm, that is leaps and bounds faster or more efficient than one in
scipy.ndimage, I would be all ears.

But I definitely don't want to get mired down in replicated code that
doesn't serve a useful purpose beyond what is already in numpy/scipy.

These comments should not in any way be construed as me passing judgement on
your code (I havent really had a chance to go through it yet). I just wanted
to make sure we are all on the same page.

Will try to go through this new code this weekend.

Cheers!

Chris

On Tue, Sep 21, 2010 at 3:57 PM, Emmanuelle Gouillart <
emmanuelle.gouillart at normalesup.org> wrote:

> > > I did not have the time to read thoroughly the code of your median
> > > filter, I think you wanted to know if this was efficient Cython code or
> > > not. I'll do it soon, maybe after you write a test function for this
> > > filter? By the way, what if the advantage of this function compared to
> > > scipy.ndimage.median_filter?
> > No there is no reason to use our own rather than scipy's one (except
> > that my implementation might be less obfuscated
> > because it is less complicated) it was more an example of what we
> > could do to bind C and python
>
> Well, clearer and simpler code is definitely an advantage :-)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20100921/7524f318/attachment.html>

From emmanuelle.gouillart at normalesup.org  Tue Sep 21 15:22:11 2010
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Tue, 21 Sep 2010 21:22:11 +0200
Subject: contributing C/Cython code to the scikit
In-Reply-To: <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com> <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
Message-ID: <20100921192211.GD31763@phare.normalesup.org>

Hi Ma??????l,

OK, so I went through almost all your changes. I made some changes myself
that you can find on
http://github.com/emmanuelle/scikits.image/tree/denoise

What I did is basically:

* cosmetic changes as described in the PEPs (spaces around operators,
  4-space indentation)

* docstrings: I modified the examples so that they could work with the
  Numpy array "lena" provided by Scipy. This avoids I/O and makes the
  examples easier to understand. 

* adding a test_denoise.py script in the tests submodule of the filter
  module. When you write functions that contain the string "test" in
  their name (e.g. test_tvdenoise), some programs (such as nosetests)
  will automatically detect and run such tests. Once again, it is better
  if you can use synthetic data or scipy's lena in these tests to avoid
  I/O.

Further changes that I may suggest:

* write a test script filter/tests/test_median.py in the same spirit of
  filter/tests/test_tvdenoise.py

* I think that 'tv_denoise' is easier to read and understand than
  'tvdenoise', maybe you should change it.

* docstrings: it's nice to provide a wikipedia page and/or the reference
  to a research article in the "References" part, so that people can
  easily learn more about the algorithm. It will especially important for
  the state-of-the-art algorithms that you have in your C library Megawave, 
  and that cannot be found elsewhere.

* if there is no good reason to keep lena256.tif (if you can use
  scipy.lena() in all your examples and tests), then why not remove the
  file.

* in the future, try to make commits that are as unitary as possible (for
  example, one commit for the tvdenoise algorithm, one for the
  corresponding test file, then for the median filter, etc.). Of course,
  it's important that the commit does not break anything (build, etc.).

I did not have the time to read thoroughly the code of your median
filter, I think you wanted to know if this was efficient Cython code or
not. I'll do it soon, maybe after you write a test function for this
filter? By the way, what if the advantage of this function compared to
scipy.ndimage.median_filter?

I also haven't gone though the C code yet. You can check PEP 7 to see if
the C code also matches the coding guidelines.

I have to say that I'm not a very experienced programmer either (at least
for large or medium-size collaborative projects), so I would be happy if
someone corrects me if I'm wrong on some points.

	Cheers,

	Emmanuelle

On Mon, Sep 20, 2010 at 10:02:16AM -0700, mael wrote:
> Hello everyone,

> here is a first code draft, including two basic filters
>  http://github.com/maelp/scikits.image

> the goal is not to include high-profile algorithms right now, but to
> get the base framework correctly (and then we'll have some engineer
> translate our codebase to python).
> could you tell me if the way I started it seems okay  with
>  you? (eg I've been hacking in some files to have everything compile..
> am I doing it as it should be done?)

> feel free to comment on code and possible  issues , as we're trying to
> have some solid base to build upon :)


From emmanuelle.gouillart at normalesup.org  Tue Sep 21 15:30:17 2010
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Tue, 21 Sep 2010 21:30:17 +0200
Subject: denoising algorithms / working with Github
In-Reply-To: <20100921160523.GB31763@phare.normalesup.org>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com> <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com> <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com> <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com> <AANLkTik33Trt9Q-Tnya0jsqUExP0FB8o4Cy3bKrGKzkK@mail.gmail.com> <20100921160523.GB31763@phare.normalesup.org>
Message-ID: <20100921193017.GE31763@phare.normalesup.org>

Just a git/github question for the future: when Mael has a set of files
that are polished enough for a pull request, will it be better to start a
new branch that is a clone of Stefan's main branch, and add these new
files to the new branch, for a cleaner commit tree? How do you proceed
usually?

Cheers,

Emmanuelle


From emmanuelle.gouillart at normalesup.org  Tue Sep 21 15:57:02 2010
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Tue, 21 Sep 2010 21:57:02 +0200
Subject: contributing C/Cython code to the scikit
In-Reply-To: <8ef1fdde-d31e-493e-adfe-55f2f3b14f42@m16g2000vbs.googlegroups.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com> <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com> <20100921192211.GD31763@phare.normalesup.org> <8ef1fdde-d31e-493e-adfe-55f2f3b14f42@m16g2000vbs.googlegroups.com>
Message-ID: <20100921195702.GF31763@phare.normalesup.org>

> > I did not have the time to read thoroughly the code of your median
> > filter, I think you wanted to know if this was efficient Cython code or
> > not. I'll do it soon, maybe after you write a test function for this
> > filter? By the way, what if the advantage of this function compared to
> > scipy.ndimage.median_filter?
> No there is no reason to use our own rather than scipy's one (except
> that my implementation might be less obfuscated
> because it is less complicated) it was more an example of what we
> could do to bind C and python

Well, clearer and simpler code is definitely an advantage :-)


From mael.primet at gmail.com  Wed Sep 22 03:54:18 2010
From: mael.primet at gmail.com (mael)
Date: Wed, 22 Sep 2010 00:54:18 -0700 (PDT)
Subject: denoising algorithms / working with Github
In-Reply-To: <AANLkTikR6SrBvwbJhdM5QbMNXck53j+1C=zxMe53h74a@mail.gmail.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
 <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com>
 <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com>
 <AANLkTik33Trt9Q-Tnya0jsqUExP0FB8o4Cy3bKrGKzkK@mail.gmail.com>
 <20100921160523.GB31763@phare.normalesup.org> <20100921193017.GE31763@phare.normalesup.org>
 <AANLkTikR6SrBvwbJhdM5QbMNXck53j+1C=zxMe53h74a@mail.gmail.com>
Message-ID: <8c27424f-4837-4fe0-b3fc-bcceda66beca@k30g2000vbn.googlegroups.com>

Hello everyone,

St?fan: actually I saw the data images, but the Lena's in it are too
small to be of use on image processing algorithms :) But Emmanuelles
scipy.lena trick
 will do the work.

Chris: this is actually an interesting discussion to have, there are
pros and cons about doing again what is done by scipy.
First, although scipy's version is more elaborate and potentially more
optimised, it is also more obfuscated, and thus more difficult to
understand. This may not be a problem for "end-users", but this could
become problematic
 when you are a researcher writing a paper and having to be certain
about everything you do. Moreover we would like the library to have a
triple goal: be used by "end-users" (people needing basic image
processing algos ),
be used by students (and therefore have clear and accessible
algorithms, not something like scipy.ndimage complex C code), and be
used by researchers (thus be easy to extend and fiddle with, and have
a complete control
over the algorithms). This is why we feel that in some case, recoding
the algorithms in a simple way could be interesting. Of course, we
won't reimplement the FFT or linear algebra algorithms..

This could be nice to discuss those points before we add too much code
(but let's be clear: most of the algos we want to add cannot be found
in scipy. Actually, the only ones would be median and morphology,
which you already implement)


From mael.primet at gmail.com  Wed Sep 22 05:56:55 2010
From: mael.primet at gmail.com (mael)
Date: Wed, 22 Sep 2010 02:56:55 -0700 (PDT)
Subject: denoising algorithms / working with Github
In-Reply-To: <AANLkTin0c0hyPJTae2B2XAXJ_+4XGeo4eJkd3pzvuJnO@mail.gmail.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
 <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com>
 <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com>
 <AANLkTik33Trt9Q-Tnya0jsqUExP0FB8o4Cy3bKrGKzkK@mail.gmail.com>
 <20100921160523.GB31763@phare.normalesup.org> <20100921193017.GE31763@phare.normalesup.org>
 <AANLkTikR6SrBvwbJhdM5QbMNXck53j+1C=zxMe53h74a@mail.gmail.com>
 <8c27424f-4837-4fe0-b3fc-bcceda66beca@k30g2000vbn.googlegroups.com> <AANLkTin0c0hyPJTae2B2XAXJ_+4XGeo4eJkd3pzvuJnO@mail.gmail.com>
Message-ID: <895f5e6b-7e26-4303-a0bb-68b6d21cd0b8@m16g2000vbs.googlegroups.com>

> While I agree with some of these sentiments, we have limited
> man-power; it seems to make more sense to focus on algorithms that
> aren't available (also, the scipy team can do bugfixing on ndimage,
> whereas any code we include becomes our responsibility).
Certainly, but something is also of value for researchers using the
library: knowing what the algorithms
exactly do (which is not obvious from the scipy.ndimage module, which
doesn't describe in depths the
algorithms, and whose implementation is obfuscated). But we will
mostly focus on original algorithms

> Have you played a bit with the methods in scikits.image.morphology?
> I'd be interested to know what you think. ?I was very happy with
> Damian's code, but so far I've only used in in class demos.
Not yet but I'll make sure to comment on them

have a good  day


From stefan at sun.ac.za  Wed Sep 22 02:02:37 2010
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 22 Sep 2010 08:02:37 +0200
Subject: contributing C/Cython code to the scikit
In-Reply-To: <20100921192211.GD31763@phare.normalesup.org>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
 <20100921192211.GD31763@phare.normalesup.org>
Message-ID: <AANLkTikO48QjYRE4sQghLY0Dp7S39+eaiGgoc0WBffdR@mail.gmail.com>

On Tue, Sep 21, 2010 at 9:22 PM, Emmanuelle Gouillart
<emmanuelle.gouillart at normalesup.org> wrote:
> * docstrings: I modified the examples so that they could work with the
> ?Numpy array "lena" provided by Scipy. This avoids I/O and makes the
> ?examples easier to understand.

Also note that we ship a small collection of test images with the
scikit.  These may be found in

scikits.image.data_dir

Regards
St?fan


From stefan at sun.ac.za  Wed Sep 22 02:25:41 2010
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 22 Sep 2010 08:25:41 +0200
Subject: denoising algorithms / working with Github
In-Reply-To: <20100921193017.GE31763@phare.normalesup.org>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
 <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com>
 <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com>
 <AANLkTik33Trt9Q-Tnya0jsqUExP0FB8o4Cy3bKrGKzkK@mail.gmail.com>
 <20100921160523.GB31763@phare.normalesup.org> <20100921193017.GE31763@phare.normalesup.org>
Message-ID: <AANLkTikR6SrBvwbJhdM5QbMNXck53j+1C=zxMe53h74a@mail.gmail.com>

Hi Emmanuelle

On Tue, Sep 21, 2010 at 9:30 PM, Emmanuelle Gouillart
<emmanuelle.gouillart at normalesup.org> wrote:
> Just a git/github question for the future: when Mael has a set of files
> that are polished enough for a pull request, will it be better to start a
> new branch that is a clone of Stefan's main branch, and add these new
> files to the new branch, for a cleaner commit tree? How do you proceed
> usually?

I think there is a lot to be said for a clean commit, especially if
you can avoid large binary files (like the Lena he checked in).

So, yes, I'd create a new branch, cherry pick, merge, rebase, go crazy
until the thing looks good, and then do a pull request.

Hope that's what you were asking :)

Cheers
St?fan


From stefan at sun.ac.za  Wed Sep 22 05:44:58 2010
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 22 Sep 2010 11:44:58 +0200
Subject: denoising algorithms / working with Github
In-Reply-To: <8c27424f-4837-4fe0-b3fc-bcceda66beca@k30g2000vbn.googlegroups.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
 <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com>
 <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com>
 <AANLkTik33Trt9Q-Tnya0jsqUExP0FB8o4Cy3bKrGKzkK@mail.gmail.com>
 <20100921160523.GB31763@phare.normalesup.org> <20100921193017.GE31763@phare.normalesup.org>
 <AANLkTikR6SrBvwbJhdM5QbMNXck53j+1C=zxMe53h74a@mail.gmail.com> <8c27424f-4837-4fe0-b3fc-bcceda66beca@k30g2000vbn.googlegroups.com>
Message-ID: <AANLkTin0c0hyPJTae2B2XAXJ_+4XGeo4eJkd3pzvuJnO@mail.gmail.com>

On Wed, Sep 22, 2010 at 9:54 AM, mael <mael.primet at gmail.com> wrote:
> St?fan: actually I saw the data images, but the Lena's in it are too
> small to be of use on image processing algorithms :) But Emmanuelles
> scipy.lena trick
> ?will do the work.

That's intentional.  We want to limit the image sizes, but have enough
data for our unit test suite!

> Chris: this is actually an interesting discussion to have, there are
> pros and cons about doing again what is done by scipy.
> First, although scipy's version is more elaborate and potentially more
> optimised, it is also more obfuscated, and thus more difficult to
> understand. This may not be a problem for "end-users", but this could
> become problematic
> ?when you are a researcher writing a paper and having to be certain
> about everything you do. Moreover we would like the library to have a
> triple goal: be used by "end-users" (people needing basic image
> processing algos ),
> be used by students (and therefore have clear and accessible
> algorithms, not something like scipy.ndimage complex C code), and be
> used by researchers (thus be easy to extend and fiddle with, and have
> a complete control
> over the algorithms). This is why we feel that in some case, recoding
> the algorithms in a simple way could be interesting. Of course, we
> won't reimplement the FFT or linear algebra algorithms..

While I agree with some of these sentiments, we have limited
man-power; it seems to make more sense to focus on algorithms that
aren't available (also, the scipy team can do bugfixing on ndimage,
whereas any code we include becomes our responsibility).

Luckily, there is an easy way to motivate inclusion: make sure it's
faster, more robust or has some other interesting feature not present
in SciPy (and that's not all that hard to achieve!).

> (but let's be clear: most of the algos we want to add cannot be found
> in scipy. Actually, the only ones would be median and morphology,
> which you already implement)

Have you played a bit with the methods in scikits.image.morphology?
I'd be interested to know what you think.  I was very happy with
Damian's code, but so far I've only used in in class demos.

Regards
St?fan


From stefan at sun.ac.za  Wed Sep 22 07:00:53 2010
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 22 Sep 2010 13:00:53 +0200
Subject: denoising algorithms / working with Github
In-Reply-To: <895f5e6b-7e26-4303-a0bb-68b6d21cd0b8@m16g2000vbs.googlegroups.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
 <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com>
 <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com>
 <AANLkTik33Trt9Q-Tnya0jsqUExP0FB8o4Cy3bKrGKzkK@mail.gmail.com>
 <20100921160523.GB31763@phare.normalesup.org> <20100921193017.GE31763@phare.normalesup.org>
 <AANLkTikR6SrBvwbJhdM5QbMNXck53j+1C=zxMe53h74a@mail.gmail.com>
 <8c27424f-4837-4fe0-b3fc-bcceda66beca@k30g2000vbn.googlegroups.com>
 <AANLkTin0c0hyPJTae2B2XAXJ_+4XGeo4eJkd3pzvuJnO@mail.gmail.com> <895f5e6b-7e26-4303-a0bb-68b6d21cd0b8@m16g2000vbs.googlegroups.com>
Message-ID: <AANLkTim3fsRgkD68chLz+E9j4Fcv8Ov4LGphO3bZ+GuB@mail.gmail.com>

On Wed, Sep 22, 2010 at 11:56 AM, mael <mael.primet at gmail.com> wrote:
> exactly do (which is not obvious from the scipy.ndimage module, which
> doesn't describe in depths the
> algorithms, and whose implementation is obfuscated). But we will

So true; no-one knows what goes on in there :)

Cheers
St?fan


From emmanuelle.gouillart at normalesup.org  Wed Sep 22 13:24:44 2010
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Wed, 22 Sep 2010 19:24:44 +0200
Subject: denoising algorithms / working with Github
In-Reply-To: <AANLkTikR6SrBvwbJhdM5QbMNXck53j+1C=zxMe53h74a@mail.gmail.com>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com> <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com> <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com> <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com> <AANLkTik33Trt9Q-Tnya0jsqUExP0FB8o4Cy3bKrGKzkK@mail.gmail.com> <20100921160523.GB31763@phare.normalesup.org> <20100921193017.GE31763@phare.normalesup.org> <AANLkTikR6SrBvwbJhdM5QbMNXck53j+1C=zxMe53h74a@mail.gmail.com>
Message-ID: <20100922172444.GA10153@phare.normalesup.org>

> I think there is a lot to be said for a clean commit, especially if
> you can avoid large binary files (like the Lena he checked in).

> So, yes, I'd create a new branch, cherry pick, merge, rebase, go crazy
> until the thing looks good, and then do a pull request.

> Hope that's what you were asking :)

Basically, yes :-). I think it will take me a lot of time before I really
understand how to use git, which always seem to be more clever than I am,
but I also think it's really worth learning. The page
http://stefanv.github.com/scikits.image/gitwash/development_workflow.html
is really good, I wish I had read it earlier. Then I would not have
pushed work-in-progress commits to my master branch on github, now I have
to find a way to clean all this mess. But that's how you learn, don't
you :-D ?

Cheers,

Emmanuelle


From stefan at sun.ac.za  Wed Sep 22 16:07:24 2010
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 22 Sep 2010 22:07:24 +0200
Subject: denoising algorithms / working with Github
In-Reply-To: <20100922172444.GA10153@phare.normalesup.org>
References: <20100909214336.GB23543@phare.normalesup.org> <b9871b3f-ba8c-4f46-8364-8c3de8875988@h25g2000vba.googlegroups.com>
 <20100910125611.GA31060@phare.normalesup.org> <c9fe0adf-4fb0-44a2-bdd5-203ebecd3d9f@a30g2000vbt.googlegroups.com>
 <AANLkTikkBeuUEUeLej_eFVR1RC3R1BTzTvD1-Fsr9z7c@mail.gmail.com>
 <e256d555-64f9-4d3b-8eda-7c5034edd935@a11g2000vbn.googlegroups.com>
 <AANLkTik33Trt9Q-Tnya0jsqUExP0FB8o4Cy3bKrGKzkK@mail.gmail.com>
 <20100921160523.GB31763@phare.normalesup.org> <20100921193017.GE31763@phare.normalesup.org>
 <AANLkTikR6SrBvwbJhdM5QbMNXck53j+1C=zxMe53h74a@mail.gmail.com> <20100922172444.GA10153@phare.normalesup.org>
Message-ID: <AANLkTi=vkwcvpa6Y_gJ2moRgC-ZVjKN+uigEFwQS_aA+@mail.gmail.com>

On Wed, Sep 22, 2010 at 7:24 PM, Emmanuelle Gouillart
<emmanuelle.gouillart at normalesup.org> wrote:
> Basically, yes :-). I think it will take me a lot of time before I really
> understand how to use git, which always seem to be more clever than I am,
> but I also think it's really worth learning. The page
> http://stefanv.github.com/scikits.image/gitwash/development_workflow.html
> is really good, I wish I had read it earlier.

Kudos to Matthew Brett for the gitwash project!
(http://github.com/matthew-brett/gitwash)

And don't feel bad about git; I managed to make it segfault today (!).

Cheers
St?fan