[scikit-learn] OMP ended prematurely due to linear dependence in the dictionary

[Vlad Niculae]

One possible issue I can see causing this is if X and y have different
dtypes... was this the case for you?

On Tue, Feb 14, 2017 at 8:26 PM, Vlad Niculae <zephyr14 at gmail.com> wrote:
> Hi Ben,
>
> This actually sounds like a bug in this case! At a glance, the code
> should use the correct BLAS calls for the data type you provide. Can
> you reproduce this with a simple small example that gets different
> results if the data is 32 vs 64 bit? Would you mind filing an issue?
>
> Thanks,
> Vlad
>
>
> On Tue, Feb 14, 2017 at 8:19 PM, Benjamin Merkt
> <benjamin.merkt at bcf.uni-freiburg.de> wrote:
>> OK, the issue is resolved. My dictionary was still in 32bit float from
>> saving. When I convert it to 64float before calling fit it works fine.
>>
>> Sorry to bother.
>>
>>
>>
>> On 14.02.2017 11:00, Benjamin Merkt wrote:
>>>
>>> Hi,
>>>
>>> I tried that with no effect. The fit still breaks after two iterations.
>>>
>>> If I set precompute=True I get three coefficients instead of only two.
>>> My Dictionary is fairly large (currently 128x42000). Is it even feasible
>>> to use OMP with such a big Matrix (even with ~120GB ram)?
>>>
>>> -Ben
>>>
>>>
>>>
>>> On 13.02.2017 23:31, Vlad Niculae wrote:
>>>>
>>>> Hi,
>>>>
>>>> Are the columns of your matrix normalized? Try setting `normalized=True`.
>>>>
>>>> Yours,
>>>> Vlad
>>>>
>>>> On Mon, Feb 13, 2017 at 6:55 PM, Benjamin Merkt
>>>> <benjamin.merkt at bcf.uni-freiburg.de> wrote:
>>>>>
>>>>> Hi everyone,
>>>>>
>>>>> I'm using OrthogonalMatchingPursuit to get a sparse coding of a
>>>>> signal using
>>>>> a dictionary learned by a KSVD algorithm (pyksvd). However, during
>>>>> the fit I
>>>>> get the following RuntimeWarning:
>>>>>
>>>>> /usr/local/lib/python2.7/dist-packages/sklearn/linear_model/omp.py:391:
>>>>> RuntimeWarning:  Orthogonal matching pursuit ended prematurely due to
>>>>> linear
>>>>> dependence in the dictionary. The requested precision might not have
>>>>> been
>>>>> met.
>>>>>
>>>>>   copy_X=copy_X, return_path=return_path)
>>>>>
>>>>> In those cases the results are indeed not satisfactory. I don't get the
>>>>> point of this warning as it is common in sparse coding to have an
>>>>> overcomplete dictionary an thus also linear dependency within it. That
>>>>> should not be an issue for OMP. In fact, the warning is also raised
>>>>> if the
>>>>> dictionary is a square matrix.
>>>>>
>>>>> Might this Warning also point to other issues in the application?
>>>>>
>>>>>
>>>>> Thanks, Ben
>>>>>
>>>>> _______________________________________________
>>>>> scikit-learn mailing list
>>>>> scikit-learn at python.org
>>>>> https://mail.python.org/mailman/listinfo/scikit-learn
>>>>
>>>> _______________________________________________
>>>> scikit-learn mailing list
>>>> scikit-learn at python.org
>>>> https://mail.python.org/mailman/listinfo/scikit-learn
>>>>
>>> _______________________________________________
>>> scikit-learn mailing list
>>> scikit-learn at python.org
>>> https://mail.python.org/mailman/listinfo/scikit-learn
>>
>> _______________________________________________
>> scikit-learn mailing list
>> scikit-learn at python.org
>> https://mail.python.org/mailman/listinfo/scikit-learn