is Sci_kiet-Learn the right choice for my project
Dear Sirs, I am evaluating SciKit-Learn for a new project. I am hoping to find a AI Machine Learning package that can take a large dataset of objects that have various object types and attributes. These objects are typically related to other objects, such as a server to a Wifi device, or two network routers to each other, etc. When these objects are setup data is gathered about where they are located, what settings there are, the device type, etc. With large organizations there can be thousands of these objects and tens of thousands of relationships, descriptions, settings, etc. My hope is that with machine learning we can detect when an object is missing, or configured in error, or duplicates. The question is, will SciKit-Learn help with this problem? I understand that we will have to train it to identify what to look for and then act on what was found and predicted to be the solution algorithm. Or instructions. Thanks for your help, Great looking product and already have the tutorial up and running and have installed it in my Django platform. Mike
Mebbe Tensor Flow would be a better fit to your needs. On 10/08/2022 1:58:03 AM, Mike Oliver <mo@globalsaassol.com> wrote: Dear Sirs, I am evaluating SciKit-Learn for a new project. I am hoping to find a AI Machine Learning package that can take a large dataset of objects that have various object types and attributes. These objects are typically related to other objects, such as a server to a Wifi device, or two network routers to each other, etc. When these objects are setup data is gathered about where they are located, what settings there are, the device type, etc. With large organizations there can be thousands of these objects and tens of thousands of relationships, descriptions, settings, etc. My hope is that with machine learning we can detect when an object is missing, or configured in error, or duplicates. The question is, will SciKit-Learn help with this problem? I understand that we will have to train it to identify what to look for and then act on what was found and predicted to be the solution algorithm. Or instructions. Thanks for your help, Great looking product and already have the tutorial up and running and have installed it in my Django platform. Mike
Dear Mike, Just my two cents about your inquiry, where I strictly a user of scikit-learn for many years. - From your description of application context, I would say that scikit-learn is perfectly fine. However, I would suggest the awareness that a monolithic model incorporating all data (as is the image TV wrongfully projects) is not a valid strategy. Stratifying data into contextually correct subgroups and then running scikit-learn, for example to estimate during development the extent of predictability, will be helpful. - Duplicate checking should be easy to use using standard python objects (set or list counting), once the context derives how the objects are vectorized/featurized. I don't see a need to force scikit-learn for that context. - Missing data could be implemented by context-specific object classes that you design, which could contain something like a __bool__() method that could tell if you if the object has all of the required data populated and configured. - Detection of errors in configuration could be either explicitly driven by logic (of the context, again something to return a bool that an object is configured correctly), or potentially could be statistically derived as outliers from the given background data distribution, in which then scikit-learn could be of help. If there are too many variates (thousands or tens of thousands) in your data that prohibit explicit logic, then scikit-learn's Random Forest algorithms might be perfectly fine and provide verification through visualization of Decision Tree rules. Hope this helps, J.B. Brown 2022年10月8日(土) 10:59 Mike Oliver <mo@globalsaassol.com>:
Dear Sirs,
I am evaluating SciKit-Learn for a new project. I am hoping to find a AI Machine Learning package that can take a large dataset of objects that have various object types and attributes. These objects are typically related to other objects, such as a server to a Wifi device, or two network routers to each other, etc. When these objects are setup data is gathered about where they are located, what settings there are, the device type, etc.
With large organizations there can be thousands of these objects and tens of thousands of relationships, descriptions, settings, etc. My hope is that with machine learning we can detect when an object is missing, or configured in error, or duplicates.
The question is, will SciKit-Learn help with this problem? I understand that we will have to train it to identify what to look for and then act on what was found and predicted to be the solution algorithm. Or instructions.
Thanks for your help,
Great looking product and already have the tutorial up and running and have installed it in my Django platform.
Mike _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
My hope is that with machine learning we can detect when an object is missing, or configured in error, or duplicates.
These look like simple correctness issues that I'd address with programming. Why do you want to use a learned approach? Do you think it will be faster to develop, or have a faster runtime? Bill -- Phobrain.com On 2022-10-08 01:57, Mike Oliver wrote:
Dear Sirs,
I am evaluating SciKit-Learn for a new project. I am hoping to find a AI Machine Learning package that can take a large dataset of objects that have various object types and attributes. These objects are typically related to other objects, such as a server to a Wifi device, or two network routers to each other, etc. When these objects are setup data is gathered about where they are located, what settings there are, the device type, etc.
With large organizations there can be thousands of these objects and tens of thousands of relationships, descriptions, settings, etc. My hope is that with machine learning we can detect when an object is missing, or configured in error, or duplicates.
The question is, will SciKit-Learn help with this problem? I understand that we will have to train it to identify what to look for and then act on what was found and predicted to be the solution algorithm. Or instructions.
Thanks for your help,
Great looking product and already have the tutorial up and running and have installed it in my Django platform.
Mike _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
Bill, Granted, you are correct the examples I gave are simple enough for a set of rules to process. The problem as I see it is that with the plethora of potential relationships, the rules may not be adequate. Inconsistent data is not transactional for example I can store data on a pair of objects and they pass all the rules. Then later another relationship is stored, and that passes all the rules…yet just because A is consistent with B and B is consistent with C does NOT mean A is consistent with C. Anomaly detection can be simple or it can be complex. But if 10,000 relationships have been stored and one changes I want an algorithm to emerge that we can see and turn into an action to fix it. We are also hoping that things may emerge that we did not anticipate. That may require deep learning sub-symbolic neural networks, but that is yet to be determined. And yes, with literally thousands of records flowing through the system, the delay in processing each record against every other record in a read before write model is not going to perform well. If we can use machine learning we can take that evaluation and even corrections out of that processing flow. Thanks, If scikit-learn is not a good fit for my goals, let me know. If you know a better fit, please let me know as well. Mike From: Bill Ross <bross_phobrain@sonic.net> Sent: Sunday, October 9, 2022 12:46 AM To: Scikit-learn mailing list <scikit-learn@python.org> Cc: scikit-learn <scikit-learn-bounces+bross_phobrain=sonic.net@python.org>; Mike Oliver <mo@globalsaassol.com> Subject: Re: [scikit-learn] is Sci_kiet-Learn the right choice for my project [Warning]CAUTION: This email originated from outside of the organization. Do not click on links or open attachments unless you recognize the sender and know that the contents are safe.
My hope is that with machine learning we can detect when an object is missing, or configured in error, or duplicates.
These look like simple correctness issues that I'd address with programming. Why do you want to use a learned approach? Do you think it will be faster to develop, or have a faster runtime? Bill -- Phobrain.com On 2022-10-08 01:57, Mike Oliver wrote: Dear Sirs, I am evaluating SciKit-Learn for a new project. I am hoping to find a AI Machine Learning package that can take a large dataset of objects that have various object types and attributes. These objects are typically related to other objects, such as a server to a Wifi device, or two network routers to each other, etc. When these objects are setup data is gathered about where they are located, what settings there are, the device type, etc. With large organizations there can be thousands of these objects and tens of thousands of relationships, descriptions, settings, etc. My hope is that with machine learning we can detect when an object is missing, or configured in error, or duplicates. The question is, will SciKit-Learn help with this problem? I understand that we will have to train it to identify what to look for and then act on what was found and predicted to be the solution algorithm. Or instructions. Thanks for your help, Great looking product and already have the tutorial up and running and have installed it in my Django platform. Mike _______________________________________________ scikit-learn mailing list scikit-learn@python.org<mailto:scikit-learn@python.org> https://mail.python.org/mailman/listinfo/scikit-learn
účastníci (4)
-
Bill Ross -
Bob Helmbold -
Brown J.B. -
Mike Oliver