Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Yup

You can trace the origins of the ANN to the 40s and 50s. Since then, statistical ML has come into vogue a few times, but not like today. Phones and other data sources (plus the algorithms to properly deal with all the data) have ushered in our current renaissance.



Huh. Why do you single out phones as a data source? That seems like a weird statement to make. Neural nets are in vogue because everyone has a phone?


Everybody has a camera in their pocket that takes pictures for free. That's increased the number of images that you can train a neural network on (and the number of images that you might want to classify with it) dramatically.

I remember when I was a kid, you had huge binders of photo albums, and the total number of pictures that a family might own numbered in the hundreds. Now I take that many photos on a week-long vacation, or sometimes just in ordinary life, and my cloud backup has maybe tens of thousands. Multiply that by a billion cell phones and that's a lot of images out there.


Data was never the issue for neural nets. It was the compute power required to run them. The most popular image recognition dataset today was mostly taken pre-smartphones.


When I was in school, studying ML, we were always able to beat NN performance with SVMs on the datasets available. It's only through larger training sets that NNs start to shine.


NNs will still be beat by svms on simple datasets, even big ones. Where NNs shine is when the data has an underlying structure that can be exploited. Like how convolutions take advantage of the 2 dimensional nature of images, or RNNs take advantage of sequential data. Algorithmic improvements like dropout and proper weight initialization have also made them bunch better than they used to be.


If you know what the structure is, then you could write a proper kernel and then SVMs are better again. NNs win when you don't know what the structure is and it's too complex to approximate with a standard kernel.


I'm not aware of anything like a convolutional SVM, but I'm not terribly familiar with them. As I understand it SVMs are fundamentally shallow learners, and if you try to hack them to make them deep and recurrent or convolutional, you just get weird NNs.


Shallow in the sense that SVMs are not layered, but the layering of a NN is only to enable the humble perceptron (logistic or sigmoid, usually) to model more complex functions. In contrast, the SVM doesn't need to layer, because the kernel can be arbitrarily complex.


Machine learning and neural networks are in vogue because it's easier than ever before to collect data for them to be trained upon and process. Phones are a part of that for sure.




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: