Mailman 3 June 2018 - scikit-learn

One-hot encoding
by Sarah Wait Zaranek Aug. 3, 2018

Aug. 3, 2018

Hello - I was just wondering if there was a way to improve performance on the one-hot encoder. Or, is there any plans to do so in the future? I am working with a matrix that will ultimately have 20 million categorical variables, and my bottleneck is the one-hot encoder. Let me know if this isn't the place to inquire. My code is very simple when using the encoder, but I cut and pasted it here for completeness. enc = OneHotEncoder(sparse=True) Xtrain = enc.fit_transform(tiledata) Thanks, Sarah

3 20

Supervised prediction of multiple scores for a document
by Amirouche Boubekki July 3, 2018

July 3, 2018

Héllo, I started a natural language processing project a few weeks ago called wikimark <https://github.com/amirouche/wikimark/> (the code is all in wikimark.py <https://github.com/amirouche/wikimark/blob/master/wikimark.py#L1>) Given a text it wants to return a dictionary scoring the input against vital articles categories <https://en.wikipedia.org/api/rest_v1/page/html/Wikipedia%3AVital_articles%2…>, e.g.: out = wikimark("""Peter Hintjens wrote about the relation between technology and culture. Without using a scientifical tone of state-of-the-art review of the anthroposcene antropology, he gives a fair amount of food for thought. According to Hintjens, technology is doomed to become cheap. As matter of fact, intelligence tools will become more and more accessible which will trigger a revolution to rebalance forces in society.""") for category, score in out: print('{} ~ {}'.format(category, score)) The above program would output something like that: Art ~ 0.1 Science ~ 0.5 Society ~ 0.4 Except not everything went as planned. Mind the fact that in the above example the total is equal to 1, but I could not achieve that at all. I am using gensim to compute vectors of paragraphs (doc2vev) and then submit those vectors to svm.SVR in a one-vs-all strategy ie. a document is scored 1 if it's in that subcategory and zero otherwise. At prediction time, it goes though the same doc2vec pipeline. The computer will score *each paragraph* against the SVR models of wikipedia vital article subcategories and get a value between 0 and 1 for *each paragraph*. I compute the sum and group by subcategory and then I have a score per category for the input document It somewhat works. I made a web ui online you can find it at https://sensimark.com where you can test it. You can directly access the full api e.g. https://sensimark.com/api/v0?url=http://www.multivax.com/last_question.html… The output JSON document is a list of category dictionary where the prediction key is associated with the average of the "prediction" of the subcategories. If you replace &all=1 by &top=5 you might get something else as top categories e.g. https://sensimark.com/api/v0?url=http://www.multivax.com/last_question.html… <https://sensimark.com/api/v0?url=http://www.multivax.com/last_question.html…> or https://sensimark.com/api/v0?url=http://www.multivax.com/last_question.html… I wrote "prediction" with double quotes because the value you see, is the result of some formula. Since, the predictions I get are rather small between 0 and 0.015 I apply the following formula: value = math.exp(prediction) magic = ((value * 100) - 110) * 100 In order to have values to spread between -200 and 200. Maybe this is the symptom that my model doesn't work at all. Still, the top 10 results are almost always near each other (try with BBC <http://www.bbc.com/> articles on https://sensimark.com . It is only when a regression model is disqualified with a score of 0 that the results are simple to understand. Sadly, I don't have an example at hand to support that claim. You have to believe me. I just figured looking at the machine learning map <http://scikit-learn.org/stable/tutorial/machine_learning_map/> that my problem might be classification problem, except I don't really want to know what is *the* class of new documents, I want to how what are the different subjects that are dealt in the document based on a hiearchical corpus; I don't want to guess a hiearchy! I want to now how the document content spread over the different categories or subcategories. I quickly read about multinomal regression, is it something do you recommend I use? Maybe you think about something else? Also, it seems I should benchmark / evaluate my model against LDA. I am rather noob in terms of datascience and my math skills are not so fresh. I more likely looking for ideas on what algorithm, fine tuning and some practice of datascience I must follow that doesn't involve writing my own algorithm. Thanks in advance!

2 3

NEP: Random Number Generator Policy
by Robert Kern July 2, 2018

July 2, 2018

https://mail.python.org/pipermail/numpy-discussion/2018-June/078126.html Hi, sklearners! I have a NEP out for discussion that proposes a change in numpy.random's stream-compatibility policy. As scikit-learn is a well-disciplined consumer of reproducible streams, I would appreciate your input on the numpy-discussion thread linked above. The very short form is that there is a new PRNG subsystem being developed with better core PRNGs (among other things, providing nice features like independent streams for parallel computations), and we would like to relax our strict stream-compatibility policy for the non-uniform distributions in this new subsystem so that we can improve our algorithms. The core uniform numbers would still be strictly stream-compatible across numpy versions. But we would like to be able to upgrade our non-uniform algorithms, for example, to make normal variates faster to generate. RandomState would be frozen and subject to a long deprecation cycle for a period of strict backwards compatibility. There would be some non-deprecated provision to get strictly-compatible streams for a subset of distributions for the limited purpose of generating test data for unit tests. Please read the NEP and the thread through. I do propose at least one alternative in the thread and would like some feedback on it. I would also appreciate it if we could consolidate the discussion on the numpy-discussion thread and not have a split-off conversation here too. Thank you very much! I appreciate your attention. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

4 9

New core dev: Joris Van den Bossche
by Olivier Grisel June 26, 2018

June 26, 2018

Hi everyone! Let's welcome Joris Van den Bossche (@jorisvdbossche) officially as a scikit-learn core developer! Joris is one of the maintainers of the pandas project and recently contributed many new great PRs to scikit-learn (notably the ColumnTransformer and a refactoring of the categorical variable preprocessing tools). Cheers! -- Olivier

5 4

Re: [scikit-learn] Scikit Multi learn error.
by Piotr Szymański June 26, 2018

June 26, 2018

Scikit-multilearn features a larger variety of models, many of which are still not above the selectiveness threshold of scikit-learn. In general, scikit-learn implements only three multi-label classifiers - BinaryRelevance, OneVsRest and ClassifierChains. And generally sklearn.multioutput is a very recent addition (2016), added 3 years after the scikit-multilearn library was started. (responding again, after joining list, i'm sorry if anyone got this twice) wt., 26 cze 2018 o 19:59 użytkownik Piotr Szymański <niedakh(a)gmail.com> napisał: > Scikit-multilearn features a larger variety of models, many of which are > still not above the selectiveness threshold of scikit-learn. In general, > scikit-learn implements only three multi-label classifiers - > BinaryRelevance, OneVsRest and ClassifierChains. And generally > sklearn.multioutput is a very recent addition (2016), added 3 years after > the scikit-multilearn library was started. > > wt., 26 cze 2018 o 19:40 użytkownik Fernando Marcos Wittmann < > fernando.wittmann(a)gmail.com> napisał: > >> Why there's a library based on Sklearn for multi classification? Sklearn >> itself can handle this ( >> http://scikit-learn.org/stable/modules/multiclass.html) >> >> On Tue, Jun 26, 2018 at 5:08 PM, Vlad Niculae <zephyr14(a)gmail.com> wrote: >> >>> Hi Aijaz, >>> >>> You're writing to the wrong mailing list. This is the mailing list for >>> scikit-learn, not scikit-multilearn, which is a different and unrelated >>> project. You're unlikely to get an answer here; >>> I recommend following the contact information on the scikit-multilearn >>> website. >>> >>> Best of luck, >>> >>> Yours >>> Vlad >>> >>> On Tue, Jun 26, 2018, 11:04 aijaz qazi <aqsdmcet(a)gmail.com> wrote: >>> >>>> Dear developer , >>>> >>>> I am working on web page categorization with http://scikit.ml/ . >>>> >>>> >>>> *Question*: I am not able to execute MLkNN code on the link >>>> http://scikit.ml/api/classify.html. I have installed py 3.6. >>>> >>>> I found scipy versions not compatible with scikit.ml 0.0.5. >>>> >>>> Which version of scipy would work with scikit.ml 0.0.5. >>>> >>>> Kindly let me know. I will be grateful. >>>> >>>> >>>> *Regards,* >>>> *Aijaz A.Qazi * >>>> _______________________________________________ >>>> scikit-learn mailing list >>>> scikit-learn(a)python.org >>>> https://mail.python.org/mailman/listinfo/scikit-learn >>>> >>> >>> _______________________________________________ >>> scikit-learn mailing list >>> scikit-learn(a)python.org >>> https://mail.python.org/mailman/listinfo/scikit-learn >>> >>> >> >> >> -- >> >> Fernando Marcos Wittmann >> MS Student - Energy Systems Dept. >> School of Electrical and Computer Engineering, FEEC >> University of Campinas, UNICAMP, Brazil >> +55 (19) 987-211302 >> >>

1 0

Scikit Multi learn error.
by aijaz qazi June 26, 2018

June 26, 2018

Dear developer , I am working on web page categorization with http://scikit.ml/ . *Question*: I am not able to execute MLkNN code on the link http://scikit.ml/api/classify.html. I have installed py 3.6. I found scipy versions not compatible with scikit.ml 0.0.5. Which version of scipy would work with scikit.ml 0.0.5. Kindly let me know. I will be grateful. *Regards,* *Aijaz A.Qazi *

3 2

Re: [scikit-learn] New core dev: Joris Van den Bossche
by Jason Wolosonovich June 25, 2018

June 25, 2018

Welcome Joris! - Jason ________________________________ From: scikit-learn <scikit-learn-bounces+jason.wolosonovich=infusionsoft.com(a)python.org> on behalf of scikit-learn-request(a)python.org <scikit-learn-request(a)python.org> Sent: Saturday, June 23, 2018 9:00 AM To: scikit-learn(a)python.org Subject: scikit-learn Digest, Vol 27, Issue 24 Send scikit-learn mailing list submissions to scikit-learn(a)python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scikit-learn or, via email, send a message with subject or body 'help' to scikit-learn-request(a)python.org You can reach the person managing the list at scikit-learn-owner(a)python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of scikit-learn digest..." Today's Topics: 1. New core dev: Joris Van den Bossche (Olivier Grisel) 2. Re: New core dev: Joris Van den Bossche (Sebastian Raschka) 3. Re: New core dev: Joris Van den Bossche (Vlad Niculae) ---------------------------------------------------------------------- Message: 1 Date: Sat, 23 Jun 2018 12:42:27 +0200 From: Olivier Grisel <olivier.grisel(a)ensta.org> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: [scikit-learn] New core dev: Joris Van den Bossche Message-ID: <CAFvE7K61Usim7Qpegc+CCNfh01xcFVLqbPTAs5ozPM-O9AkZaA(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hi everyone! Let's welcome Joris Van den Bossche (@jorisvdbossche) officially as a scikit-learn core developer! Joris is one of the maintainers of the pandas project and recently contributed many new great PRs to scikit-learn (notably the ColumnTransformer and a refactoring of the categorical variable preprocessing tools). Cheers! -- Olivier

1 0

imbalanced classes: class_weight
by S Hamidizade June 21, 2018

June 21, 2018

Hi I would appreciate if you could let me know what is the best way to categorize the approaches which have been developed to deal with imbalance class problem? *This article <https://www.sciencedirect.com/science/article/pii/S0020025513005124> categorizes them into:* 1. Preprocessing: includes oversampling, undersampling and hybrid methods, 2. Cost-sensitive learning: includes direct methods and meta-learning which the latter further divides into thresholding and sampling, 3. Ensemble techniques: includes cost-sensitive ensembles and data preprocessing in conjunction with ensemble learning. *The second <https://dl.acm.org/citation.cfm?id=2907070> classification:* 1. Data Pre-processing: includes distribution change and weighting the data space. One-class learning is considered as distribution change. 2. Special-purpose Learning Methods 3. Prediction Post-processing: includes threshold method and cost-sensitive post-processing 4. Hybrid Methods: *The third article <https://link.springer.com/article/10.1007/s13748-016-0094-0>:* 1. Data-level methods 2. Algorithm-level methods 3. Hybrid methods The last classification also considers output adjustment as an independent approach. Could you please let me know the class-weight in the sklearn's classifiers e.g., logistic regression is classified into which category? Is it true to say: In case of the first categorization, it falls into cost-sensitive learning In case of the second taxonomy, it would be classified into the third category i.e., cost-sensitive post-processing In case of the third classification, it should fall into algorithm level Best regards,

3 4

Mapping fulltext OCR to issue type
by Thomas Güttler June 18, 2018

June 18, 2018

We run an issue tracking application. A lot of issues get generated from scanned letters. I have 70k full text OCR result files. Their got created with tesseract. Every file of these 70k files corresponds to a issue. Each issue has an issue type. I want to use machine learning and in the future the machine should be able to guess the issue type by looking at the full text OCR. The issue types are not a simple list, it is a tree. Example: electricity / power grid electricity / outages customer support / invoices / complaint customer support / invoices / tax .... If the machine can't guess "customer support / invoices / complaint" it would be nice if it could at least guess roughly the parent issue type: "customer support / invoices" I never used sciki before, but I use Python since several years. Could you please guide me to the right direction? Regards, Thomas Güttler -- Thomas Guettler http://www.thomas-guettler.de/ I am looking for feedback: https://github.com/guettli/programming-guidelines

2 3

help
by Alexandra Metallinou Log June 16, 2018

June 16, 2018

Dear Sir/Madam, I have stumbled upon a problem while trying to run some old code using skcikit-learn: scores = cross_validation.cross_val_score(model, X, Y, cv = 10, score_func = metrics.mean_squared_error) This line will not run in a program I downloaded, and as I am not yet very familliar with scikit-learn I do not know how I should replace "score_func = metrics.mean_squared_error" to produce the same result as intended by the ones who made the program. Any help is greatly appreciated. Best regards, Alexandra Log

4 5