[scikit-learn] OMP ended prematurely due to linear dependence in the dictionary

[Benjamin Merkt]

Is this still considered a bug and therefore worth an issue?

On 14.02.2017 13:34, Benjamin Merkt wrote:
> Yes, the data array y was already float64.
>
>
> On 14.02.2017 12:28, Vlad Niculae wrote:
>> One possible issue I can see causing this is if X and y have different
>> dtypes... was this the case for you?
>>
>> On Tue, Feb 14, 2017 at 8:26 PM, Vlad Niculae <zephyr14 at gmail.com> wrote:
>>> Hi Ben,
>>>
>>> This actually sounds like a bug in this case! At a glance, the code
>>> should use the correct BLAS calls for the data type you provide. Can
>>> you reproduce this with a simple small example that gets different
>>> results if the data is 32 vs 64 bit? Would you mind filing an issue?
>>>
>>> Thanks,
>>> Vlad
>>>
>>>
>>> On Tue, Feb 14, 2017 at 8:19 PM, Benjamin Merkt
>>> <benjamin.merkt at bcf.uni-freiburg.de> wrote:
>>>> OK, the issue is resolved. My dictionary was still in 32bit float from
>>>> saving. When I convert it to 64float before calling fit it works fine.
>>>>
>>>> Sorry to bother.
>>>>
>>>>
>>>>
>>>> On 14.02.2017 11:00, Benjamin Merkt wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I tried that with no effect. The fit still breaks after two
>>>>> iterations.
>>>>>
>>>>> If I set precompute=True I get three coefficients instead of only two.
>>>>> My Dictionary is fairly large (currently 128x42000). Is it even
>>>>> feasible
>>>>> to use OMP with such a big Matrix (even with ~120GB ram)?
>>>>>
>>>>> -Ben
>>>>>
>>>>>
>>>>>
>>>>> On 13.02.2017 23:31, Vlad Niculae wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Are the columns of your matrix normalized? Try setting
>>>>>> `normalized=True`.
>>>>>>
>>>>>> Yours,
>>>>>> Vlad
>>>>>>
>>>>>> On Mon, Feb 13, 2017 at 6:55 PM, Benjamin Merkt
>>>>>> <benjamin.merkt at bcf.uni-freiburg.de> wrote:
>>>>>>>
>>>>>>> Hi everyone,
>>>>>>>
>>>>>>> I'm using OrthogonalMatchingPursuit to get a sparse coding of a
>>>>>>> signal using
>>>>>>> a dictionary learned by a KSVD algorithm (pyksvd). However, during
>>>>>>> the fit I
>>>>>>> get the following RuntimeWarning:
>>>>>>>
>>>>>>> /usr/local/lib/python2.7/dist-packages/sklearn/linear_model/omp.py:391:
>>>>>>>
>>>>>>> RuntimeWarning:  Orthogonal matching pursuit ended prematurely
>>>>>>> due to
>>>>>>> linear
>>>>>>> dependence in the dictionary. The requested precision might not have
>>>>>>> been
>>>>>>> met.
>>>>>>>
>>>>>>>   copy_X=copy_X, return_path=return_path)
>>>>>>>
>>>>>>> In those cases the results are indeed not satisfactory. I don't
>>>>>>> get the
>>>>>>> point of this warning as it is common in sparse coding to have an
>>>>>>> overcomplete dictionary an thus also linear dependency within it.
>>>>>>> That
>>>>>>> should not be an issue for OMP. In fact, the warning is also raised
>>>>>>> if the
>>>>>>> dictionary is a square matrix.
>>>>>>>
>>>>>>> Might this Warning also point to other issues in the application?
>>>>>>>
>>>>>>>
>>>>>>> Thanks, Ben
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> scikit-learn mailing list
>>>>>>> scikit-learn at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/scikit-learn
>>>>>>
>>>>>> _______________________________________________
>>>>>> scikit-learn mailing list
>>>>>> scikit-learn at python.org
>>>>>> https://mail.python.org/mailman/listinfo/scikit-learn
>>>>>>
>>>>> _______________________________________________
>>>>> scikit-learn mailing list
>>>>> scikit-learn at python.org
>>>>> https://mail.python.org/mailman/listinfo/scikit-learn
>>>>
>>>> _______________________________________________
>>>> scikit-learn mailing list
>>>> scikit-learn at python.org
>>>> https://mail.python.org/mailman/listinfo/scikit-learn
>> _______________________________________________
>> scikit-learn mailing list
>> scikit-learn at python.org
>> https://mail.python.org/mailman/listinfo/scikit-learn
>>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn