Just signed up for Claude Pro to try out the Opus model. Decided to throw a complex query at it, combining an image with an involved question about SDXL fine tuning and asking it to do some math comparing the cost of using an RTX 6000 Ada vs an H100.
It made a lot of mistakes. I provided it with a screenshot of Runpod's pricing for their GPUs, and it misread the pricing on an RTX 6000 ADA as $0.114 instead of $1.14.
Then, it tried to do math, and here is the outcome:
-----
>Approach 1: Use the 1x RTX 6000 Ada with a batch size of 4 for 10,000 steps.
You will note that .278 * $0.114 (or even the actually correct $1.14) != $19.00, and that .116 * $4.69 != $19.54.
For what it's worth, ChatGPT 4 correctly read the prices off the same screenshot, and did math that was more coherent. Note, it saw that the RTX 6000 Ada was currently unavailable in that same screenshot and on its own decided to substitute a 4090 which is $.74/hr, also it chose the cheaper PCIe version of the H100 Runpod offers @ $3.89/hr:
-----
>The total cost for running 10,000 steps on the RTX 4090 would be approximately $2.06.
>It would take about 2.78 hours to complete 10,000 steps on the RTX 4090.
On the other hand:
>The total cost for running 10,000 steps on the H100 PCIe would be approximately $5.40.
>It would take about 1.39 hours to complete 10,000 steps on the H100 PCIe, which is roughly half the time compared to the RTX 4090 due to the doubled batch size assumption.
I'm convinced GPT is running separate helper functions on input and output tokens to fix the 'tokenization' issues. As in, find items of math, send it to this hand made parser and function, then insert result into output tokens. There's no other way to fix the token issue.
I'd almost say anyone not doing that is being foolish.
The goal of the service is to answer complex queries correctly, not to have a pure LLM that can do it all. I think some engineers feel that if they are leaning on an old school classically programed tool to assist the LLM, it's somehow cheating or impure.
> I'd almost say anyone not doing that is being foolish
The problem is, such tricks are sold as if there's superior built-in multi-modal reasoning and intelligence instead of taped up heuristics, exacerbating the already amped up hype cycle in the vacuum left behind by web3.
Why is this a trick or somehow inferior to getting the AI model to be able to do it natively?
Most humans also can’t reliably do complex arithmetic without the use of something like a calculator. And that’s no trick. We’ve built the modern world with such tools.
Why should we fault AI for doing what we do? To me, training the AI use a calculator is not just a trick for hype, it’s exciting progress.
By all means if it works to solve your problem, go ahead and do it.
The reason some people have mixed feelings about this because of a historical observation - http://www.incompleteideas.net/IncIdeas/BitterLesson.html - that we humans often feel good about adding lots of hand-coded smarts to our ML systems reflecting our deep and brilliant personal insights. But it turns out just chucking loads of data and compute at the problem often works better.
20 years ago in machine vision you'd have an engineer choosing precisely which RGB values belonged to which segment, deciding if this was a case where a hough transform was appropriate, and insisting on a room with no windows because the sun moves and it's totally throwing off our calibration. In comparison, it turns out you can just give loads of examples to a huge model and it'll do a much better job.
(Obviously there's an element of self-selection here - if you train an ML system for OCR, you compare it to tesseract and you find yours is worse, you probably don't release it. Or if you do, nobody pays attention to you)
I agree we should teach our AI models how to do math, but that doesn’t mean they shouldn’t use tools as well.
Certain problems are always going to be very algorithmic and computationally expensive to solve. Asking an LLM to multiply each row in a spreadsheet by pi for example would be a total waste.
To handle these kinds of problems, the AI should be able to write and execute its own code for example. Then save the results in a database or other long term storage.
Another thing it would need is access to realtime data sources and reliable databases to draw on data not in the training set. No matter how much you train a model, these will still be useful.
The reason we chucked loads of data at it was because we had no other options. If you wanted to write a function that classified a picture as a cat or a dog, good luck. With ML, you can learn such a function.
That logic doesn’t extend to things we already know how to program computers to do. Arithmetic already works. We don’t need a neural net to also run the calculations or play a game of chess. We have specialized programs that are probably as good as we’re going to get in those specialized domains.
Not so fast - you might have precise and efficient functions that do things like basic arithmetic. What you might not have is a model that can reason mathematically. You need a model to do things like basic arithmetic functions so that semantic and arbitrary relations get encoded in the weights of a network.
You see this type of glitch crop up in tokenizing schemes in large language models. If you attempt working with character level reasoning or output construction, it will often fail. Trying to get ChatGPT 4 to output a sentence, and then that sentence backwards, or every other word spelled backwards, is almost impossible. If you instead prompt the model to produce an answer with a delimiter between every character, like #, also to replace spaces, it can resolve the problems much more often than with standard punctuation and spaces.
The idea applies to abstractions that aren't only individual tokens, but specific concepts and ideas that in turn serve as atomic components of higher abstractions.
In order to use those concepts successfully, the model has to be able to encode the thing and its relationships effectively in the context of whatever else it learns. For a given architecture, you could do the work and manually create the encoding scheme for something like arithmetic, and it could probably be very efficient and effective. What you miss is the potential for fuzzy overlaps in the long tail that only come about through the imperfect, bespoke encodings learned in the context of your chosen optimizer.
Damn, how many problems with LLMs relate to the encoding of the token? Surely every symbolic manipulation task is getting thrown off by this. Memorizing the multiplication table of two three digit numbers is no easy task at all. That explains why the interpreter hack works so well. The python interpreter sees things digit by digit, but the LLM does it token by token.
I've asked it so many times to count the number of words or letters and it was incredibly bad at it.
Since it is capable of splitting large tokens into smaller tokens, the solution to this problem is to create additional training samples that perform "big token" to "small token" conversion and back, so that the model will learn to dynamically provide the most suitable encoding to itself.
> We don’t need a neural net to also run the calculations or play a game of chess.
That's actually one of the specific examples from the link I mentioned:-
> In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. At the time, this was looked upon with dismay by the majority of computer-chess researchers who had pursued methods that leveraged human understanding of the special structure of chess. When a simpler, search-based approach with special hardware and software proved vastly more effective, these human-knowledge-based chess researchers were not good losers. They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. These researchers wanted methods based on human input to win and were disappointed when they did not.
While it's true that they didn't use an LLM specifically, it's still an example of chucking loads of compute at the problem instead of something more elegant and human-like.
Of course, I agree that if you're looking for a good game of chess, Stockfish is a better choice than ChatGPT.
What was considered “loads of compute” in 1998 is the kind of thing that can run on anyone’s phone today. Stockfish is extremely cheap compared with an LLM. Even a human-like model like Maia is tiny compared with even the smallest LLMs used these services.
Point is, LLM maximalists are wrong. Specialized software is better in many places. LLMs can fill in the gaps, but should hand off when necessary.
It would be exciting if the LLM knew it needed a calculator for certain things and went out and got it. If the human supervisors are pre-screening the input and massaging what the LLM is doing that is a sign we don't understand LLMs enough to engineer them precisely and can't count on them to be aware of their own limitations, which would seem to be a useful part of general intelligence.
It can if you let it, that's the whole premise of LangChain style reasoning and it works well enough. My dumb little personal chatbot knows it can access a Python REPL to carry out calculations and it does.
Because if NN is smart enough, it should be able to do arithmetic flawlessly. Basic arithmetic doesn't even require that much intelligence, it's mostly attention to detail.
Well it’s obviously not smart enough so the question is what do you do about it? Train another net that’s 1000x as big for 99% accuracy or hand it off to the lowly calculator which will get it right 100% of the time?
And 1000x is just a guess. We have no scaling laws about this kind of thing. It could be a million. It could be 10.
I agree with you that we don't know if will take 10x or 1 million. We don't know if current LLM will scale at all. It might not be the way to AGI.
But while we can delegate the math to the calculator, it's essentially sweeping the problem under the rug. It actually tells you your neural net is not very smart. We know for a fact that it was exposed to tons of math during training, and it still can't do even the most basic addition reliably, let alone multiplication or division.
What we want is an actually smart network, not a dumb search engine that knows a billion factoids and quotes, and that hallucinates randomly.
Maybe I'm too corporate-pilled, but if the 'taped up heuristics' provide noticeably better performance for real-world problems, then I don't really care that there is a facade layer around the model itself. In fact, I would pay for that difference in intentional design/optimization if one vendor does it much better than another for my use case.
I took an artificial neutral network class at the university back in 2009. On the exam we were asked to design a (hardware) system to solve a certain complex problem, then present it to the professor. The professor was actually a biologist specialised in neurology who had veered off into ANN without understanding electronics nor programming.
I recognised that the problem, while being beyond what an ANN could do at the time, could be split into two parts each of which was a classic ANN task. For communication between the two I described a very simple electronic circuit - just a few logic gates.
When presenting the design, the professor questioned why this component was not also a neutral network. Thinking it was a trick question, I happily answered that solving it that way would be stupid since this component was so simple and building and training another network to approximate such a simple logical function is just a waste of time and money. He got really upset, saying that is how he would have done it. He ended up giving me a lower score than expected saying I technically had everything right but he didn't like my attitude.
Well, these people are not wrong per se. Scale is what drove what we have today and as hardware improves, the models will too. It's just that in the very short term it turns out to be faster to just code around some of these issues on the backend of an API rather than increase the compute you spend on the model itself.
We're rapidly approaching the compute capacity of the human brain in individual server racks. This "moon" is neither unreachable nor is there any doubt that we will cross the threshold soon.
I find it incredibly hard to believe we stumbled upon an efficient architecture that requires nothing but more compute not 10 years after the AI winter thawed. That's incredibly optimistic to the point of blind hope. What is your background and what makes you think we've somehow already figured everything out?
I have been working on architectures in this field for almost a decade now and I've seen firshand how things have changed. It might seem hard to believe if you have been to university ~10 years ago and only know the state of deep learning from the early revolutions back then, but we are in a totally different era now. With the transformer, we now have a true general-purpose, efficiently scalable, end-to-end differentiable algorithm. Meaning you can apply it to any task as long as you convert it to the right embedding space, you can train gigantic models that compress huge amounts of information using enormous datasets and you can still use good-ol' gradient descent to optimize it (which is kind of sad since we still haven't found a better way of training models, but hey it works).
> Meaning you can apply it to any task as long as you convert it to the right embedding space
This glosses over a massive issue which is that not everything can be efficiently represented as a vector space via embeddings. So your claim of "general purpose" rings hollow.
Not to mention that there is no feedback mechanism for the supposed "knowledge" advocates claim Transformer-based models have, so things like metacognition are literally impossible with this architecture. As it stands LLM outputs are isomorphic to psychotic stream-of-consciousness babble.
You've managed to find a tall tree, but from your response it seems like you haven't yet gotten to considering rockets.
There is no basis to any of your arguments. Embedding spaces are not defined by humans, but learned. A priori you have zero idea in what way or how efficient things will be encoded. It all depends on the model and it's internally learned world representation. And things like "metacognition" are meaningless terms used by quacks. We don't know how the brain works, we only know what it can do on the outside. And it is mathematically proven that neural networks can do literally everything in principle as well, thanks to universal approximation.
and yet SamA says it's actual trillions of dollars in entirely new compute capacity to reach the next level. hmmm. to believe him or you... so hard to decide.
SamA is a business bro who tries to hoard investor capital. He has to say these things to collect more VCs. If you want to learn about the tech, listen to what the actual techies at openai have to say. This stuff is no secret.
"We" contains more than just one perspective though.
As someone applying LLMs to a set of problems in a production application, I just want a tool that solves the problem. Today, that tool is an LLM, tomorrow it could be anything. If there are ~hacks~ elegant techniques that can get me the results I need faster, cheaper, or more accurately, I absolutely will use those until there's a better alternative.
Like a AGI? I think we’ll put up with hacks for some more time still. Unless the model gets really really good at generalizing and then it’s probably close to human level already
It's not cheating or impure. It's a path that is not pointing towards AGI. Heterogenous architectures are seen with contempt nowadays. Everyone told us that it is better to have a single huge model, than to specialize anything at all.
I personally find approaches like this the correct way forward.
An input analyzer that finds out what kinds of tokens the query contains. A bunch of specialized models which handle each type well: image analysis, OCR, math and formal logic, data lookup,sentiment analysis, etc. Then some synthesis steps that produce a coherent answer in the right format.
Yeah. Have a multimodal parser model that can decompose prompts into pieces, generate embeddings for each of them and route those embeddings to the correct model based on the location of the embedding in latent space. Then have a "combiner/resolver" model that is trained to take answer embeddings from multiple models and render it in one of a variety of human readable formats.
Eventually there is going to be a model catalog that describes model inputs/outputs in a machine parseable format, all models will use a unified interface (embedding in -> embedding out, with adapters for different latent spaces), and we will have "agent" models designed to be rapidly fine tuned in an online manner that act as glue between all these different models.
Doesn't the human brain work like this? Yeah it's all connected together and plastic and so on, but functions tend to be localized, e.g vision is in occipital area. These base areas are responsible for the basic latent representations (edge detectors) which get fed forward to the AGI module (prefrontal cortex) that coordinates the whole thing based on the high quality representations it sees from these base modules.
This strikes me as the most compute efficient approach.
That has nothing to do with the idea of ensembling multiple specialized/single-purpose models. Mixture of Experts is an method of splitting the feed-forwards in a model such that only a (hopefully) relevant subset of parameters is run for each token.
The model learns how to split them on its own, and usually splits based not on topic or domain, but on grammatical function or category of symbol (e.g., punctuation, counting words, conjunctions, proper nouns, etc.).
ChatGPT definitely has a growing bag of tricks like that.
When I use analysis mode to generate and evaluate code it recently started writing the code, then introspecting it and rewriting the code with an obvious hidden step asking "is this code correct". It made a huge improvement in usability.
Fairly recently it would require manual intervention to fix.
GPT has for some time output "analyzing" in a lot of contexts. If you see that, you can go into settings and tick "always show code when using data analyst" and you'll see that it does indeed construct Python and run code for problems where it is suitable.
Regardless of emergence, in the context of "putting safety at the frontier" I would expect Claude 3 to be augmented with very basic tools like calculators to minimize such trivial hallucinations. I say this as someone rooting for Anthropic.
An "LLM crawler app" is needed -- in that you should be able to shift Tokenized Workloads between executioners in a BGP routing sort of sense...
Least cost routing of prompt response. especially if time-to-respond is not as important as precision...
Also, is there a time-series ability in any LLM model (meaning "show me this [thing] based on this [input] but continually updated as I firehose the crap out of it"?
--
What if you could get execution estimates for a prompt?
What a joke of a response. No one is asking for emergent calculation ability just that the model gives the correct answer. LLM tools (functions etc) is old news at this point.
When OpenAI showed that GPT-4 with vision was smarter than GPT-4 without vision, what did they mean really? Does vision capability increase intelligence even in tasks that don't involve vision (no image input)?
I cant wait until this is the true disruptor in the economy: "Take this $1,000 and maximise my returns and invest it where appropriate. Goal is to make this $1,000 100X"
And just let your r/wallStreetBets BOT run rampant with it...
It made a lot of mistakes. I provided it with a screenshot of Runpod's pricing for their GPUs, and it misread the pricing on an RTX 6000 ADA as $0.114 instead of $1.14.
Then, it tried to do math, and here is the outcome:
-----
>Approach 1: Use the 1x RTX 6000 Ada with a batch size of 4 for 10,000 steps.
>Cost: $0.114/hr * (10,000 steps / (4 images/step * 2.5 steps/sec)) = $19.00 Time: (10,000 steps / (4 images/step * 2.5 steps/sec)) / 3600 = 0.278 hours
>Approach 2: Use the 1x H100 80GB SXMS with a batch size of 8 for 10,000 steps.
>Cost: $4.69/hr * (10,000 steps / (8 images/step * 3 steps/sec)) = $19.54 Time: (10,000 steps / (8 images/step * 3 steps/sec)) / 3600 = 0.116 hours
-----
You will note that .278 * $0.114 (or even the actually correct $1.14) != $19.00, and that .116 * $4.69 != $19.54.
For what it's worth, ChatGPT 4 correctly read the prices off the same screenshot, and did math that was more coherent. Note, it saw that the RTX 6000 Ada was currently unavailable in that same screenshot and on its own decided to substitute a 4090 which is $.74/hr, also it chose the cheaper PCIe version of the H100 Runpod offers @ $3.89/hr:
-----
>The total cost for running 10,000 steps on the RTX 4090 would be approximately $2.06.
>It would take about 2.78 hours to complete 10,000 steps on the RTX 4090. On the other hand:
>The total cost for running 10,000 steps on the H100 PCIe would be approximately $5.40.
>It would take about 1.39 hours to complete 10,000 steps on the H100 PCIe, which is roughly half the time compared to the RTX 4090 due to the doubled batch size assumption.
-----