The more I get into machine learning and deep learning it seems like there is an incredible amount of configuration to get some decent results. Cleaning and storing the data takes a long time. And then you need to figure out exactly what you want to predict. If you predict some feature with any sort of error in your process the entire results will be flawed.
There are a few very nice applications of the AI techniques, however most data sets don't fit well with machine learning. What you see is that in tutorials use the Iris data set so much because it breaks into categories very easily. In the real world, most things are in a maybe state rather than yes/no.
> In the real world, most things are in a maybe state rather than yes/no.
Not to get too far afield, but I disagree with this on a certain philosophical level. All states are yes/no. All states of all things should result in a yes/no and be differentiable, with enough data. This doesn't speak to the practicality of that but as far as I can tell the theoretical potential is huge, almost infinite even.
But there should be no ambiguity, with enough data. Maybe that means there will always be ambiguity, but maybe it doesn't, especially not with man-made things and complex natural objects, and also if you can contextualize the data over time and 'geographically', there is more 'signal' there to differentiate
> But there should be no ambiguity, with enough data.
What? I'm sorry but this runs counter to everything in my experience, both professionally, and just casual very day experience.
More data, helps to a point, but then there's diminishing returns, and it certainly doesn't eliminate the ambiguity. On the contrary, you discover diversity, and you still have a misclassification and perhaps even a harder data cleaning problem, because now you're seeing cases that aren't actually clear cut. Even if you're only talking about adding more features, well again, that works up to a point, but then you hit sparsity issues.
Yeah that's why I called it philosophical, because the idea is a little more involved, shall we say. I'm not a god of this, so speculation ahead bewarned. In cases that aren't clear cut, you would also need contextual data like bigger actual physical area, or over time dimension, really any data point that can help narrow down what the thing is. It wouldn't just be pure deep learning stuff, it would be some kind of memory and data store of already classified objects and contexts. In the ultimate end, ALL of it would be sparse, but classify perfectly just that one thing it is built to classify. And if that doesn't work, several sparse things combined would result in one unique thing. On the sparsity matrix wikipedia page there is an example of balls with a string through them, this would correspond to the data being the balls and the systems we build (or alternatively unsupervised learning methods for finding new strings), whatever they may be, would be the strings (assuming all the strings are actual informational and correct to natural world). But you need the balls to begin with etc. Since all of this information should be in the natural world by its own, and also accessible to us
This is literally a philosophical problem. It's called ontology. And no amount of data solves this problem, because ultimately it's a labeling problem, and the border between things is ill defined, and additional data doesn't help resolve labeling ambiguity, if anything it finds out just how ill defined the world actually is.
Think about it. Let's say you had a problem which was find the black squares. So you collect some data and you find that you have a whole bunch of squares that are on the blackness scale of 0.0, and bunch that are 0.1, and then there's one at 0.5. Is 0.5 black? Maybe not. What about 0.7? Maybe. What about 0.999? Probably, but is it? It's not 1.0. And if we say 0.9 and higher are black, why not 0.89? Even discounting measurement error, there's nothing that supports a threshold at 0.9 beyond, "Well, I think it should be this."
> if anything it finds out just how ill defined the world actually is.
Yeah I hear this but it seems only half-true to me. While for most intents and purposes the world is ill-defined, in another sense the world itself is "100% signal" and no noise. If we "zoom out" and take a grand view, imagining that we have a supercomputer and a huge database, and the algorithms are solved, I think every 'thing' in the universe has some unique features, and if you start to have them all in a database you may be able to uniquely identify any thing, at least those important to us. Everything one has excludes something else, but it also includes that specific thing. Every thing adds context to one thing and removes context from another. If you can draw a map of it, it seems to me like deep learning can, hypothetically, automatically differentiate it. Deep learning isn't just about one vector or one hierarchy of features, it's about how the world is ALL vectors like this, even if right now, the CS around it is pretty limited. It seems to me intuitively true at least. At the bare minimum, seeing as us humans are absurd about categorizing everything into objects, and it actually works very well functionally (we can manipulate, create and predict in the world)
If I understand your point, I'd suggest that it may apply best to the use of DL for low level AI -- seeing, hearing/generating speech, and recognizing/ navigating/ interpreting complex signals of other kinds. There classification is secondary to modeling the many subsymbolic facets endemic to raw analog signals.
I suspect DL will eventually settle into a less vaunted role in the historical saga of AI than it portends now. And that role may well be the 'grounding' of sensory experience -- the modeling of the world into something perceptually and cognitively manageable, like Plato's shadows on a cave wall.
What I mean is the Iris data set splits into 3 categories easily.
Right now, they are saying AI self driving cars can get their predictions right 95%+ of the time. However, the cases where they cannot classify the object is the problem. Those are the "maybe" cases I was referring to where they algorithm simply cannot classify the object no matter how much it has seen.
Nah. There is a difference in my mind between creating a 'map' of the natural world, and deciding which actions to perform in that world, and this is all theoretical/philosophical. If one were to do a pure AI type programming language it would have to come from the AI itself, and it would need some reason to create that language and who knows where/what all those rules would be and why (think of human programmers who make new languages).
Your average data-set does not fit into 3 nice categories like the Iris data-set does. For example, with the Iris data set if you know the sepal length and the petal length, you can say with near certainty which type of flower it is. Even trying to classify other objects in nature is much harder than this dataset.
Now let's take sentiment analysis which tries to determine if some words are positive or negative. If someone said: 'that new machine learning algorithm is so sick'. The algorithm has no way of knowing that sick may be slang to mean good, because the system looks up 'sick' and finds that is a negative word. Sentiment analysis has no way of defining sarcasm or other natural language terms.
> Sentiment analysis has no way of defining sarcasm or other natural language terms.
Of course it can. If humans are capable of detecting a given inflection, computers absolutely can as well (given enough data).
Any sentiment analysis algorithm which classifies "that new machine learning algorithm is so sick" as negative is not worth an ounce of consideration. Compared to other problems, that is absolutely trivial to classify, especially since you're typically training off data sets which already include such vernacular.
There are a few very nice applications of the AI techniques, however most data sets don't fit well with machine learning. What you see is that in tutorials use the Iris data set so much because it breaks into categories very easily. In the real world, most things are in a maybe state rather than yes/no.