Anyone setting out to do something like this will quickly see how much of a hassle it is to exfiltrate data from relevant services and your own devices. APIs and SDKs cover a lot of use-cases, and that is all great and commendable, but we don't truly own our own devices, data, or the algorithms that make them useful. I think this will continue to hamper our ability to have a truly useful 3rd party (your home in this case) orchestrate your digital life.
A worry about reading about Zuckerberg digging into this area is that anything Facebook does will likely be within their walled garden. There is certainly a glimmer of hope they'll make something on the "open" side, but I wouldn't get my hopes up.
I wonder to what extent having great advances in this area by proprietary cloud services will hamper the advancement and adoption of libre alternatives.
> My experience of ramping up in the Facebook codebase is probably pretty similar to what most new engineers here go through. I was consistently impressed by how well organized our code is, and how easy it was to find what you're looking for -- whether it's related to face recognition, speech recognition, the Messenger Bot Framework [messenger.com/platform] or iOS development. The open source Nuclide [github.com/facebook/nuclide] packages we've built to work with GitHub's Atom make development much easier. The Buck [buckbuild.com] build system we've developed to build large projects quickly also saved me a lot of time. Our open source FastText [github.com/facebookresearch/fastTex...] AI text classification tool is also a good one to check out, and if you're interested in AI development, the whole Facebook Research [github.com/facebookresearch] GitHub repo is worth taking a look at.
Face recognition and speech recognition are the most obvious pieces that look to be proprietary, but there is lots of other useful open stuff that he explicitly links. FastText looks like it would be really helpful for at least the natural language piece of Jarvis-like applications.
I haven't really dug into open source speech recognition, so I can't speak much to that side of things, except observing that advancements favor deep learning and cloud-based services over efficient on-device detection (aside from very specific "hey siri"/"okay google"/"hey alexa" recognizers). When it's not a library that can be included, but a service that must be maintained, it'll be that much hard for free and open solutions to get adoption.
For face detection & recognition, some of the greatest advancements are getting acqui-hired and locked away (e.g. Facebook acquiring Face.com) there is still progress being made with tools like OpenCV & Dlib. I just worry progress in the open-source space will lag behind and continue to be difficult to work with, as the delta grows between proprietary capabilities and open-source options.
A worry about reading about Zuckerberg digging into this area is that anything Facebook does will likely be within their walled garden. There is certainly a glimmer of hope they'll make something on the "open" side, but I wouldn't get my hopes up.
I wonder to what extent having great advances in this area by proprietary cloud services will hamper the advancement and adoption of libre alternatives.