I wonder how big the Pro model is that Google is using behind the scenes to train these smaller ones.
Going on baseless speculation, the lack of accompanying pro models with these flash releases either means: 1) the model is too big to be economical, 2) google doesn't have the compute to serve the big model, 3) their big model has too many alignment issues to serve to the public.
edit: looks like benchmarks are up on https://artificialanalysis.ai/models/gemini-3-6-flash. It's solidly middle-of-pack. However, if you want to be most fair to flash, look at the intelligence vs time per task and intelligence vs outputspeed benchmarks. This is a very fast model.
edit 2: I use antigravity from time to time and in my experience, 3.5 flash is an underrated model, so long as you know what it's good for. It's very good at frontend (much better than gpt 5.5) and it's fast, so it's a great tool for iteration. I expect 3.6 to be no different.
It's also very possible that they know their big model underperforms chatgpt 5.6 and fable by too much, so they are focusing on what they can get wins in like speed instead.
That and/or the business case isn’t as clear when serving enormous models? You’re constantly stuck in a red queen’s race where your profitability window is increasingly measured in weeks because the Chinese are right behind you.
For small models (which are probably distilled from their big ones) you can serve them economically all the time and not hemorrhage money.
The Chinese have been right behind OpenAI and Anthropic for ~18 months now.
DeepSeek didn't do to OpenAI and Anthropic what nearly everybody claimed they would.
Every single person on HN that loudly proclaimed the end was nigh for GPT & Co. due to DeepSeek, was wrong. They were humiliatingly wrong, and they'll never own up to it. The reason those people were so very wrong, is the same exact reason the Kimi crowd is wrong now. And it's very obvious that they're wrong, but they have intense emotional blinders on. Their thinking process is hyper emotionalism: they want a certain outcome, regardless of if reality aligns to that or not. They're making emotional wishes about how they want things to turn out, and pretending those magic wishes are grounded in reason.
It takes enormous resources to run something equivalent to GPT 5.6 or Fable. Nobody can or wants to do that outside of very limited situations - if you can just reasonably pay as you go instead. As it turns out, you can just pay as you go with GPT and Fable. Their businesses have gotten radically larger since DeepSeek launched. Get it yet?
Domestic China is the only very large audience for their own models, so long as OpenAI and Anthropic stay top tier.
All the hype online from the forums about Kimi, is worthless: those people hyping it can't even come close to running it locally, which is the fantasy. So why are they hyping it? Why did they hype DeepSeek just the same, and learn nothing from its total failure to actually take down OpenAI and Anthropic? Rhetorical questions with obvious answers.
Kimi poses zero actual threat to OpenAI and Anthropic. Those companies will continue to pile up the subscriptions and API usage. Check out GPT's subscriber base today vs when DeepSeek launched. Get it yet? When Model X launches out of China in a year, we'll have this same conversations all over again, and the hypsters will have learned nothing.
While the Kimi fawning is endless, OpenAI will just keep piling up subscriber counts, and Anthropic will keep piling up API usage. Then OpenAI is going to staple a gigantic ad system onto GPT. China can't compete in the model-as-a-service business globally, for the exact same reason they failed so miserably to compete in search globally.
without any hard data one way or another your comment is worthless. "pile up subscriptions" - based on what? neither company is public. "piling up subscriber counts", "piling up API usage"? cool. how much money are they making? oh you don't know because they're not public.
the reality is one way or another that as long as there exists an alternative that a USA company could serve with the same compute rented from hyperscalers, this represents a threat, even if the extent to which is unknown
My view then was they are optimising the models for inference ability on their own hardware AND use cases, which is often speed and time to first token.
They've somehow seemed to end up with terrible compute shortages, which again is surprising given how good Google is at infra deployments AND have their own hardware. From rumors out there they are turning down enterprise deals for Gemini because they don't have the compute.
The problem is they're falling further and further behind on frontier class on coding especially, and since I wrote that article it's got even worse with open weights models undercutting them on price AND intelligence.
There was some recent reporting that a July release of the Pro model got pushed back for exactly that reason. Its performance was not good compared to the OpenAI/Anthropic big models. They are having a lot of problems with posttrain.
> focusing on what they can get wins in like speed instead
Speed as a differentiator has always been Google's thing. They (used to?) show the microseconds it took to query & rank web-scale search results. Chrome, notoriously, focused on speed at the expense of resource use. The very many efforts to efficiently speed up Android & its runtime since its inception, and so on...
> their big model underperforms chatgpt 5.6
Possible but TFA claims:
We have started our most ambitious pre-training run yet, for Gemini 4 ...
It would be a shame if they cannot beat Kimi K3 or Qwen3.8 Max, both of which are claimed to be Fable-like. If that is true, it will be [or would be] the first time a major American lab falls behind a Chinese competitor.
I wonder if the broad use of AI overviews on Google search results is having an impact. Maybe the numbers make it more profitable to use their compute on several billion searches a day rather than selling API access.
AI overview is just a summarization of the top 2-3 results. Of course at Google scale that will still need a ton of compute, but the requirement for generating an overview is many orders of magnitude lower than asking the same question in Gemini.
It’s a small multilingual embedding model designed for things like search, RAG, and semantic similarity. It supports a fairly large context window and is designed to run efficiently on a CPU in a GPU starved world.
The interesting part is that it builds on BitNet, using ternary weights of -1, 0, and 1 instead of the usual floating-point weights. That should make indexing and searching large amounts of text much cheaper without giving up too much accuracy.
It seems like there are some credible rumors that Google is actually winning in terms of actually building models that work and don't lose money- between how they're able to price them, the TPU advantage and their capex advantage (being able to raise debt + just having a lot of cash - well I said not lose money... more like not go bankrupt).
From the outside they look like they're behind in terms of frontier models, but I think they might be the best positioned to not go out of business when the bubble pops.
Also look at the fact that they've been able to deploy AI-assisted search at google scale. It must be another order of magnitude larger (at least) than the model deployments for OpenAI and Anthropic.
Of course unless you're inside Google it's impossible to know for sure.
It's rumored that Gemini 3.5 flash has a >50% margin, and I'd imagine 3.6 flash is even higher.
I do not think OpenAI or Anthropic are actively chasing margins - though, Anthropic is supposed to be profitable on some form of non-GAAP accounting...
I suspect Google isn't really interested in seeing how far it can get dragged into a race of selling dollars for $0.25, and is more interested to see if it can stay in the race selling $0.50 for a dollar - when everyone else is losing or barely breaking even.
In terms of open models, Gemma 4 beats the pants off everything else to the point that paying for APIs becomes hard to justify. Qwen has the meme-share for coding, but it feels much less well rounded. I have no doubt that Google have both the infrastructure and the expertise to curb stomp everyone else, should they resolve in earnest to do so.
Lest we forget, "Attention is All You Need" came from Google.
"As of October [2025], OpenAI's compute margins reached 70%, up from 52% at the end of 2024 and double the rate in January 2024, [The Information] said, citing a person familiar with the figures."
As for Anthropic, the rumors I remember seeing for their API margins were more like 85-90%, but I don't have a reference at hand for those. But once you know the API is wildly profitable and the subscriptions are roughly break-even and not even a big slice of their income, all of the investment makes a lot more sense.
Google somehow managed to snatch defeat from the jaws of success with their AI products.
They literally forced me and my company out of Antigravity by phasing out AI Ultra subscription without any proper product follow-up. Antigravity IDE cannot even have poweruser subscriptions now from Google Workspace an Gemini Enterprise Agent Platform cannot be attached to Antigravity IDE.
Gemini Enterprise Agent Platform has an incredibly abysmal setup process, and if I want to limit spending per-user I have to create projects per user. The fact that you cannot activate Anthropic models on it if the billing still has free credits is almost a joke.
I was a big proponent of Google and Gemini, but they left us reeling with their abrupt product decisions. Forced us to buy $200 subscriptions directly from Anthropic/OpenAI.
Also a big proponent of Google and Gemini, but their stubbornness in artificially splitting their consumer and enterprise products is extremely annoying. It's pretty weird that I have access to more powerful tools when using my personal Google account compared to my corporate Google Workspace account.
It's literally the meme of the MS org chart pointing guns at each other.
The GCP team wants their slice, the other team wants some otjer slice, and so on. Everyone wants some crap for their promotion package.
It's no wonder Meta has shit the bed even worse.
It's also why Google still releases actually decent, useful models despite the product being such a hilarious mess. A lot of the time Gemini models have actually been better as production LLMs as part of LLM-based production applications than OpenAI and Anthropic models when it comes to the complete cost:quality:latency:adherence picture. And they still are. We have products in prod that use Gemini because they're better than any other model at the specific task. But we wouldn't dare use it for anything coding related, or even just as productivity tool to rely on, because as a consumer product it's a joke.
> The GCP team wants their slice, the other team wants some otjer slice, and so on. Everyone wants some crap for their promotion package.
I got a Google One plan for Gemini, but it came bundled with YT Premium lite, and that somehow made it impossible to renew YT Premium for 30 days. I suspect different teams stealing customers from each other.
Google also gave away 1 year Gemini plans with Pixel phones that either did not work at all for existing Google One users or messed up subscriptions by downgrading your account to worse plan, or making your existing paid time shorter if you been on cheaper plan or recently changee countries. Etc.
Like when you try to give Google money they try to squeeze you as much as possible.
At the same time you can get 5 time more limits for free just by registering 10 free Google accounts.
Apparently you cannot turn off using your data as training data with gemini. This is in line with Google's general privacy policies and it's seeming need to create a stasi file on every human.
As someone who's been using Workspace as a personal email account for over a decade this has been such a struggle forever. Just lots of odd limitations to feature sets all over the place.
When they swapped Google Assistant for Gemini as the default voice provider in Android Auto it was so annoying. My wife's non-work space account can get Gemini to do the normal things like play music and what not, but my Workspace one can't do much of anything at all. I can talk about nearly any random topic with it, but getting it to change the playlist, nah, can't help you there.
It's no surprise to me to see them fumble actually supporting a lot of the consumer features of Gemini into Workspace.
I'm in the same situation. But I was shocked discovering it goes both ways: many new Gemini functionalities are only accessible using a consumer account instead of a Workspace account. Also, Gemini is now the only major AI assistant with no support for MCP connectors. Instead of adding this to the core product, like ChatGPT and Claude did, somebody at Google decided that it was smarter to add this fundamental feature to a new product instead: for enterprises this is Gemini Enterprise (which is a product completely different from the Gemini app); for consumers this the new Gemini Spark agent (meaning that you can use MCP within Spark but not within a "non-agentic" chat)... It's clear to me this a symptom of Google shipping their org chart, which is a disaster from a product perspective.
This drove me bonkers. You can enable play music (etc) in Android Auto for workspace accounts by enabling apps in Gemini. From memory (looking at the settings now, not 100% sure of the magic steps required), but go to admin.google.com, go to 'generative ai', 'gemini app', and 'apps settings', then turn on 'other Google apps'. This lets you play music (and other things) in Android Auto.
Ah, that could be it. I saw that "Other Google apps" and didn't think that would mean Spotify, but I guess its Android Auto or Google Assistant stuff. I'll give that a try, thanks for the tip.
Also have a workspace as a personal email and ended up getting a personal gmail just to try out the subscriptions before I gave up.
I have multiple anthropic and OpenAI max plans. For Gemini I just use my Cursor $200 a month plan (which also gives me the ability to try grok, conductor, etc)
It's absolute insanity. They have all the resources to have been able to lead from the front with this new technology. They have the products and users to integrate this technology into people's already existing lives. But they keep fumbling.
They're not even benchmarking against other models now, just against themselves - which tells you everything you need to know.
There is absolutely no loyalty when it comes to coding. Nothing could be more common than people threating to jump ship whenever another frontier or open source model comes.
Google is clearly able to keep growing their free and consumer and small business use cases. Unlike corporate coding, we actually have evidence that solo and small businesses can actually see productivity gains.
Anthropic and OpenAI need to stay dancing like mad, because it's their source revenue which underpins their investments.
Why does Google need to shove something at the top at the same desperate cadence? Other than "recursive self improvement leads to AGI" it seems perfectly fine if they push out something dramatically better every year and half.
> Google somehow managed to snatch defeat from the jaws of success
This is still very early days. Who is "on top" has flipped back and forth many times already. The next frontier model release (from whomever) will change things again.
I don't think it's that early tbh, agentic coding has ~90% adoption in the US.
Claude Code has largely won individual developer mindshare and has been on top ever since it came out. The benchmarks change, but almost nobody opts to use anything other than Claude IME when I ask them. Enterprise is more competitive since they care about costs and other things, but developers leaning towards Claude puts a thumb on the scales there.
The product doesn't have much lock in, so it is possible to dislodge Claude, and Anthropic could (and some may argue is likely to) just shoot themselves in the foot again and again and again, but Google has never been particularly good at enterprise sales, and they have never actually been at the frontier of intelligence.
I think Google's incentives have mostly about building models for their products, which makes them focus more on the cheap end, and while they need that, it feels like the Innovator's Dilemma is biting them here.
I own a lot of Google stock from working there in the past and have been quite happy about their trajectory up until the last 6 months, but I am getting pretty antsy about their AI story these days.
> Claude Code has largely won individual developer mindshare and has been on top ever since it came out.
Claude Code's success is not due to the agent but because the model is considered the best for programming and is very heavily subsidized, compared to pay as you go API prices. Consumers and Enterprise are not really locked in and will go where it makes the most sense.
I think they have almost no loyalty by actual developers.
There is no reason to be loyal....There is no moat.
Basically you may choose to drink brand A water bottle, brand B water bottle or tap water. Oh and you might choose the glass water bottle if you use API/Fable.
There’s no reason to be loyal, but I guess it’s a bit like any tool, once you get used to how one works why would you change to another? There is some stickiness with an LLM + harness.
Exactly developers can switch to another coding cli and the learning curve is close to zero. Mindshare without switching costs is just a popsicle in the sun.
Every model has its strengths and weaknesses, being loyal is suboptimal unless you mean being loyal to all of them, which is why cursor would have been well positioned before it got acquired. Now you have to jump through hoops to call Gemini from Claude from codex. Yuck.
Claude code was one of the first agentic code tool and when openai release models similar in performance they didn't do as well in their tools (now codex)
Claude Code (or any other model/infra/harness co-design) is not subsidized in any normal use of that word. It's trough filling (I've written this up in lurid detail so I'm only going to do it again if anyone cares).
Early days or not, Google fucking deleted my IDE and wiped my settings. It took them days to roll out a fix, by which point I had migrated off Antigravity.
I guess? Maybe I'm alone here but I don't feel like Fable is particularly more useful than Sonnet most of the time. I feel like the LLMs are good enough for the majority of uses and the hyper expensive premium ones are way into diminishing returns. At this point with Kimi being as good as it is, if they jack up the price any more I'll just go open source.
I don't think that's true if the reason a company has left the vendor for given model by making it hard to buy. Enterprise IT is enough of a pain in the butt that people will forego the new shiny to avoid the old painful unless it's genuinely better. As you say though, the best frontier model flips regularly, so companies won't go through the hassle of deploying a model if it's proved horrible to do in the past. They'll just skip that model because their current one is fine.
Model quality is only one aspect, the bigger problem is making it work in a fully integrated enterprise platform, and Google has always been lacking when it came to the latter.
At this rate, if Google has a flagship model, you're better off plugging it into a competitor's tooling than hope Google figures out how to use it.
Aren’t the subscriptions extremely subsidized and burning cash for Anthropic and OpenAI? A reasonable explanation is they’re simply abstaining from the war of attrition, especially given cheaper comparable models are breaking the illusion that the “frontier of intelligence” has any kind of per token margin.
This is hotly debated and completely unclear. Let's say Anthropics Opus models cost the same to serve as GLM 5.2. GLM 5.2 is 4.4$/MTok while Opus is 5.6 times more expensive. Assume that GLM 5.2 is served at essentially zero margin. Then Anthropic has >80% margin on API pricing. So even if an average person with a subscription pays only 20% of the API price of their usage, Anthropic makes money on subscriptions.
And the real numbers could be better for Anthropic. It's feasible Opus models are actually cheaper to serve than GLM 5.2 because Anthropic have optimized the hell out of inference.
Sure, but then why wouldn’t I use GLM 5.2 at cost or K3?
I guess that’s the big question, will people pay a big margin long term to use their end products / models or will AI tokens be commoditized by many competing players. For coding if I had to pay API costs I’d switch in a heartbeat, enterprise maybe more reluctant?
they're subsidized if you max them out, i'd imagine most users are paying $20 for maybe $2-5 of tokens.
anthropic probably has more customers that use more of their sub, but for open ai where a lot of their subs are consumers through chatgpt.com, they have a lot of free money to work with there
Possibly, but aren't the tech giants positioned to win a war of attrition? Then again, they're more likely to sit on that cash and wait for the opportune time to buy a frontier lab.
Google Gemini agy is not allowing you to use your token via your own harness. My own harness is far more efficient than agy. They can take a simple stance - if you exceed your token limit they block you - with the 5 hr limit they are already doing this . so there should be no reason to block you from using your own harness - if you are more efficient - you gain - if you are less you lose .
They are not allowing me to hit their endpoints which agy hits - it's frustrating . i tried to hack it with gemini itself. what i love about gemini is it's so encouraging and ready to help you - even against the agy client : ) .
Even though im so frustrated with this - i still love Gemini for some reason ! Most encouraging model in the world!
> I was a big proponent of Google and Gemini, but they left us reeling with their abrupt product decisions.
Likewise. This seems like a common feel. I have at least spent $4000 and likely a lot more on Gemini API because I really wanted them to win. I gave up.
I am going to ask a very direct question and only because I am curious.
Why do you care? Why would you spend your own money to a multi trillion dollar company so that they win their own "war" against another multi trillion dollar company?
Please don't get me wrong, I know the question can seem a bit negative, I am really just curious.
No, it's a fair question. The answer: I believe(d) in Demis Hassabis's vision of AI
Although, I think saying 'wanted them to win' was not accurate. More like, I stuck with them hoping it will get better, and it did get better in many ways, coding was not one of them.
i've got to wonder how much of this is intentional, and how much of this is just google being their usual terrible selves at anything consumer-product related.
Google's biggest and most important customer for all this AI stuff is google. Do they actually want other customers, or is having other people use their AI just an annoyance at this point, where we use up compute that they'd rather use internally...
Life gets simpler and better as soon as you stop giving money to Google.
It’s so silly that individual people still use their shit. Corporations, I understand - they always choose the most mediocre stacks and tools by default. But why people choose to bring the mediocrity of Google into their lives is beyond me.
> they left us reeling with their abrupt product decisions
This is what happens when you put a McKinsey consultant in the role of CEO of an organization where product managers run the asylum instead of engineers.
It's funny that they triggered the infamous "Code Red" moment in OpenAI when the 3-3.1 models came out. I switched to using them for a lot of single-shot LLM calls because they were fast and cheap. Their only area that they were lacking in was agentic/tool calling.
Needless to say 3.5 was a disappointment. Curious to see 3.6.
> Google somehow managed to snatch defeat from the jaws of success with their AI products.
HN lives in a bubble.
I have German/Italian/Polish clients virtually all use Gemini and NotebookLM. Talking insurance, banking, consulting, legal.
The real world doesn't look at pointless benchmarks on writing react tailwind crap, they are already google suite users, get the tools, test them and adopt them, end of story.
It's going to be like with angular, never mentioned on the net, widely used in the real world.
I have friends and family who use Gemini, but entirely because their Pixel phones came with a year of it for free. No other reason, and they will most likely never pay for it.
They literally forced me and my company out of Antigravity by phasing out AI Ultra subscription without any proper product follow-up
That sounds awful.
For those of us who don't follow the AI hype cycle, what does that have to do with the topic of this thread: Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber?
I am growing tired of these pelicans posts every time a new model is published. Feels to me like low effort personal brand promotion. Just sharing my 2 cents.
+1. First thing I look for in a model announcement thread. I actually came across this one an hour ago and was sad there were no pelicans yet.
It's a decent heuristic because the better models generate better pelicans. That's all. Nobody sane is going to make a bet on a model based on a pelican. But it's cool, it's tradition by now, and it's a semblance of a good first impression for new models.
If enough people agreed with you, simonw's top level comment would be grey. It isn't. Not every comment is for every person, that is fine and normal. He didnt hijack a popular thread in here to make his post. He doesnt have a tin can in your face shouting from a soapbox that makes it hard to ignore. Flag or ignore are reasonable options that can be used.
People aren't supposed to upvote or downvote posts for these kinds of reasons here. Most people are downvoting far too much on this website, and it leads to significant echo-chamber dynamics that are worse than even reddit. The pelican test continuing to be taken seriously is a great example of that kind of echo chamber.
"He doesn't have a tin can in your face shouting from a soapbox that makes it hard to ignore."
He metaphorically does because people upvote his pelicans to the top and the ensuing comment threads are massive/bloated. Huge amounts of the readership of this website are lurkers who don't even know how to hide these giant posts. Look at how bloated this very thread is right now!
Also, a lot of people unironically are whining about him because of sour grapes. Pay them what Simon is likely making, give them as much mindshare/attention as Simon gets, and they wouldn't be so mad.
Anti-incumbency bias and anti-elitist attitudes are good actually.
> People aren't supposed to upvote or downvote posts for these kinds of reasons here.
Agreed, but scrolling past or hitting - were apparently off the table for the complainers. So flagging was yet another tool in their toolbelt and I bet a powerful one at that. If simonw's pelican posts routinely went dead from flagging, he would not make them. You know that, I know that.
> The pelican test continuing to be taken seriously is a great example of that kind of echo chamber.
You can try and support that argument if you like. But I would implore you to realize that it has been had many times recently and the other side does in fact find value and do not see it that way.
> He metaphorically does because people upvote his pelicans to the top and the ensuing comment threads are massive/bloated.
Users upvote the pelicans because they find it interesting. they arent paid trolls or simonw fanatics.
> Huge amounts of the readership of this website are lurkers who don't even know how to hide these giant posts.
They can learn... it's called hackernews. For those interested, that is what the [-] link is for above the comment. Use it and move on.
But, also, as said, you're an industry (and foss!) veteran, so I find it impossible to believe that you haven't had your fair share of baseless bullshit being thrown at you, and with that, you gaining a persona that will not be hit by that, because it clearly knows that it is in fact bullshit.
Unless of course it doesn't really know that with certainty.
As said, I would _love_ to give you the benefit of the doubt, because you might just have a stressful day or whatever, but content marketing is literally your whole thing by now.
It is impossible for me to do that with a clean conscience.
Your blog front page currently opens with
> Earlier this month I hosted a fireside chat session at the AI Engineer World’s Fair with Cat Wu and Thariq Shihipar from Anthropic’s Claude Code team.
That is not what "some rando foss maintainer we are morally obligated to be soft with" does.
I'm a professional blogger now. I still also work on open source software. I'm even fine being called an "influencer" (shudder), but I take offense to accusations of unethical behavior.
I think very hard about the ethics of what I'm doing and how I can best use my "platform" (shudder again) in as constructive a way as possible.
What do you want from him, seriously? Anyone who follows the scene knows that blogging about AI (and participating in conferences etc.) is what Simon does nowadays.
There's nothing shady here. The disclosure is front and center on his About page on his website.
He's not spamming you. It's one short link, sometimes a link to a first-impressions post. It's interesting and useful for me and the other commenters who keep upvoting his comments. Why are you so antagonistic?
But how else am I supposed to know when we've reached AGI, until I see an absolutely flawless pelican?
All of the pelicans so far have had really weird flaws / quirks so I am always a little interested to see how well these models perform at this task, since I've seen all the past pelicans and have some anchoring.
Seeing a truly flawless pelican would tell me that the model has true visual reasoning capabilities as well as good taste.
At this point, it's kind of a hackernews thing. Simon posts them as a single comment in the relevant thread. It's okay for this place to have a little bit of a sense of community, and you can just ignore the comment.
Not only does it give you a super easy-to-grok understanding of the model quality just by looking at the image, but when you compare tokens and costs (both input and output), you really get a good, simple COST x QUALITY evaluation across models.
I agree to rednb that at this point it feels like rather obvious brand building, but also, I agree with you that some value is in it.
It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
Sorry mate, but you sound jealous in all these replies that the Pelican domain isn't your gig. The below is as labored as nitpicks ever get:
> It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
I thought the Gemini 3.5 Flash Lite response was quite telling myself. I personally like the Pelican SVG test, to me it is still a charming snapshot of model performance anecdata. No one would argue it's rigorous but I don't think it was ever intended to be.
I get people burning out on the pelican SVG test alongside the rest of the AI burnout, but I guess for myself I'm just choosing to keep enjoying it while I still can.
Disagree. They're a nice tradition, but besides that, they're a useful way of eyeballing improvements. I realise labs are likely to be training for Pelicans - but if they're all training for them, the differences in the results are as indicative as they were before labs trained for them.
The 3.6 Flash pelican is just about the best I've seen.
Yeah. It's something I can do myself in a couple seconds if I want, also on more varied SVG scenes. If this is going to be a benchmark people turn to I'd like to see more effort put into it than just a one-sentence prompt.
I'll split the difference. When it's a blog post there's usually an interesting observation or two, but if it's totally automated? Maybe just do the ones with a post.
A piece of the frame is missing between pedals and back wheel. The frame of the bike passes through the bird. It also puts a cap on the bird's head, and a fish in it's mouth.
The fish and the cap where always added when I asked an llm to improve it's first attempt.
This continues the trend in LLM progress of better=more stuff
Edit: I wonder if this is a function of the reasoning training, where more tokens/ stuff is rewarded.
It's a bit disheartening to see no comparison to other models here - and I'm not sure this pushes the curve anywhere. 3.6 flash is more expensive than GLM 5.2 - but seemingly worse, although this post is really light (lite?) on details.
It seemed for a time that Google had finally gotten the ball rolling, but I'm doubting that more and more as time passes. We'll see what happens with 3.5 pro I suppose.
How does your comparison work? It places Gemini 3.6 Flash Medium above GPT 5.6 Sol High and Fable 5 Medium, which makes me skeptical because that... would be making headlines that I'm not seeing right now.
I have created various questions/tests and put the models through the same tests.
I record whether the answers are correct, and the generation stats (costs, latencies, tokens used, etc.).
I have no idea why the Gemini models do so well.
I have recently added new tests, whose sole purpose was to find some cases on which Gemini 3 Flash fails (I don't like cherry-picking models or tests, but I also find it strange Gemini Flash models leading in accuracy). I made a more complex coding/tool-usage test, that I expected it to fail, it did fail it once locally in my debug tests, but when I finalized the test and ran the entire testing suite for all models, somehow Gemini 3 Flash still got it right...
Gemini models are REALLY intelligent (and they are actually my favorite model to use via the chat app to ask questions), but they somehow fail in real-word coding tasks where they have to modify files, check results, debug, etc.
My tests harness provides a lot of mock data, and limits the number of actions a model can choose from. I am starting to think that maybe the models are not bad, just that the coding harness are not optimized for those type of models, and Google doesn't really provide their own "Codex".
Interesting, well it'd be interesting to check out some individual examples where Gemini beat the others.
Also, would be great if you could add GPT 5.6 Sol XHigh and Fable 5 High as well, just to see if at least those beat Gemini which is currently your #1.
I don't like to divulge tests, but one of them is a chess puzzle.
> would be great if you could add GPT 5.6 Sol XHigh and Fable 5 High as well
I would like too, but I avoided them for several reasons:
1) Cost - this is a hobby project, those models would cost tens of dollars for each benchmark run, multiply this by tens or hundreds of models and ...
2) Time - the high models are already taking a really long answer to respond (5-10minutes per question). I run each question with 3 repeats (run the same test three times), so it would take 30 minutes per test. If I change my tests, methodology, or add a new test, it would take a really long time to run the benchmark. Also, I like having results immediately when a new model is released, now I can post within 30 minutes of a model's release the benchmark results.
3) High reasoning usually does WORSE on most tests - if you look at the leaderboard, it's sometimes counter-intuitive, but models with high or max reasoning usually do worse than medium and low. This is because the questions are quite targeted/direct, and the models overthink the question and miss the solution. Or the long thinking context makes them perform poorly. The generation tasks (SVGs/HTML animation) are usually better with longer reasoning, but short code fixes, trivia questions, puzzles, etc. are answered by low/med reasoning with more accuracy in general
Also, Fable is borderline un-testable, it refuses to answer many questions, so it scores poorly anyway.
Gemini scores 21/22 because it answers all tests, and it does them correctly, consistently. The only failed test is I think because it miscounted the lines in a file, when responding on which line the bug was in a code snippet.
Oh, and I've also added weights to different categories, so Coding and Tool usage categories influence the score more. This done both to better account for how most people are being used, and also to reduce Gemini's dominance in general/domain specific knowledge.
So yes, Gemini models are at the top, even if I actually (not proud of it) tried to make tests that actually favour other coding-focused models.
It’s really surprising. When Apple announced the multi-billion dollar deal with Google to power Apple Intelligence I thought great things were coming. Instead we are getting more and more bad news: delayed Pro models and AI leadership leaving. I wonder if Apple know something the rest of us don’t know or if they are already regretting their decision.
Apple isn't counting on their model to be a frontier coding and cowork model. Gemini is perfectly fine for the tasks that new Siri is supposed to be doing.
Besides Apple apparently making Siri AI model agnostic, the choice to go with Google was almost certainly for practical reasons. Google is a low-risk established player that already has a long work history with Apple. Google also isn't in an existential battle to establish themselves, Gemini still amounts to just another project at Google. There is tangible non-zero risk that either OAI or Anthropic will be gone in 5 years, or will be forced to leave Apple high and dry to save themselves. There is almost no risk Google will be in either such position. And worst case scenario, Google has incredibly deep pockets should Apple pursue a "refund."
What Apple wants out of Google is Siri that runs at 8gb ram and isn’t a horrible embarrassment that feels like a primitive markov chain. Given how good Gemma 4 is, Google can squeeze some serious performance in small models. Whether they can make bleeding edge models is irrelevant to Apple.
As someone on the Apple beta.. the model is almost completely irrelevant to the experience. Apple has gone and done Apple things by nerfing the experience so completely that almost any model in the past year would be fine. I still reach for ChatGPT/Claude/Grok constantly instead of the AI toy that Apple calls the new Siri.
The one thing I've found google's models to be the best at is proofreading text in non-english languages. Probably because I imagine they have the most training data for it as Google probably has the most complete archive of the internet.
I've never questioned the word lite before because it's existed my whole life.. So does it make sense? Why does it exist and where does it come from? More than coming from "light".
3.6 Flash would be a great model at 3.0 flash pricing. At this pricing, its thoroughly trounced by about 10 models on cost/performance including Grok 4.5.
3.5 Flash-ite would be a great model at 2.5 flash-lite pricing, as is, its trounced by many models including Deepseek v4 Flash.
As is, they are thoroughly outclassed for most usecases. I will say the one area where i do see Gemini punching above its weight class is in tasks that are effectively "Google this for me" / knowledge stuff. So it does have a role, and I do use it. So while I think Google is still in a strong position overall, they are really stuck as a tier 2 AI player right now with text models. They are tier 1 in bio, images, and video.
Could also be that they are pricing it at levels where they actually make money. Without seeing the behind the scenes compute cost on all of these its hard to really judge.
That being said with any open model we of course do know the total cost (or estimate)
3.5 Flash was always too expensive for a "flash" model. They marketed it as "near frontier" level, but there are several order-of-magnitude cheaper open models that compete with it.
In my tests, 3.6 Flash is NOT more token efficient, so it actually ends up costing more than 3.5 Flash, even with the output price reduction.
EDIT:
It less less verbose in final output though, but it reasons more.
I assume the optimization comes when you have long-running tasks with many tool calls, and by reasoning more, it reduces the number of tool calls needed.
Pricing often reflects what the vendors (expects) the customer is willing to pay. It seems that Google is still trying to find their niche in the market.
I guess they meant to release Gemini 3.5 Pro shortly after 3.5 Flash, but then Mythos/Fable and later GPT-5.6 came out with higher performance than 3.5 Pro, so the managers decided not to release it.
Bottom line it looks about on equal footing with GLM 5.2 in terms of both overall intelligence and cost per task, while being significantly faster (in fact it is the fastest model on artificial analysis as of rn [0])
Basically, you avoid anything dynamic: model change, tool change, etc it's also important that your system prompt or main prompt doesn't have non-static data like the date/time/place or someone's name (the person you interact with in a chatbot for example). That should be left to tool call or search.
I just put the varying parameters in a trailer prompt and have them change every time. It doesn’t matter because the cache is prefix keyed. You lose caching for the last 20 tokens or so but that’s not a big deal. Moving it to a tool call makes it too slow (needs full roundtrip).
If you’re constructing the prompt you don’t have to jam everything together you can arrange it appropriately.
All of the models, you need to have a consistent input to get the cache hit. So if you are chatting with a document, and change the system prompt, it will be a cache miss, even if the rest of the items are all the same. If you even pass in the document in not the same order as the prompts, it will be a cache miss. Or if you add tool calls or structured outputs, it will be a cache miss. (Since those generally go at the beginning of the prompt call, not at the end.)
Most of the time when reading documents from URLs directly it will never cache. (Need to typically pass in the bytes directly, or use the provider document store index.)
Gemini has a 4096 minimum token size with the 3 version models before even getting a cache hit. OpenAI it is lower (1024), and is automatic, but only happens in increments of 124. Anthropic can also get cache hits at 1024 tokens, but you need to explicit ask for it (and pay extra).
Caching by default typically lives for 5 minutes since the last cache hit across providers. But some of them you can ask for longer. AWS for Anthropic models can be tricky with multiple endpoint routing, so can get cache misses if it happens to route to a different endpoint.
That’s part of why, since Firebase, I’ve tried to never depend on Google products for business, especially not GCP.
Features stay in Beta for ages, whatever that actually means, and released ones get deprecated things fast.
Where some of the competitions treats deprecating entire services as "let’s not put it on your frontpage, put deprecation notices all over the doc, and politely ask new users not to start new project with them".
its not always that simple. dropping in a new model is trivial, but highly specific workflows may rely on specific _invisible_ aspects of a model. when that model gets deprecated, the workflow needs to be rebuilt/re-tuned to work with a different model.
google's inability or unwillingness to provide stable timelines for model deprecation makes it risky to build complex workflows using their models
You would be surprised how much of a difference the model makes for certain niche tasks.
For my use case, `gemini-3.1-flash-lite` is ~20% higher accuracy than the next best model of comparable cost (considering both proprietary and open-weight alternatives)
Well it is a bit surprising that 3.1 flash-lite could be better than deepseek-v4-pro (cheaper output and way cheaper cache so might cost less for quite a few use cases).
They are not anywhere close according to pretty much every benchmark (even v4-flash is considerably ahead and its way cheaper than flash-lite). Maybe tuning prompts/tools/etc. might be useful?
"Intelligence" being what, math? Coding? Unfortunately there's a billion use cases for LLMs whose performance is not at all captured by the popular benchmarks they're all trying to maxx.
They know that there's big enterprises that will have a strong preference to work with another big enterprise instead of relying on a younger company. At least that's why I think they believe they can do this sort of thing and get away with it.
There are plenty of 3rd party providers hosting deepseek models, if you don't want to use the 1st party API. 3rd party providers are generally slightly more expensive, but still quite cheap compared to other models of similar vintage and size.
same here. our production workloads was on Gemini for 2 years. seeing Google unilaterally dropping perfectly fine models and charing you 50x more for worse results is not good.
I'm running price-sensitive data extraction workloads on flash 2.5 and its still the king when it comes to accuracy + cost, all the gemini 3 variants perform a bit worse and cost a lot more. Low-key freaking out, ngl
I felt the same way about openai's text-davinci-002 and code-davinci-002 (gpt-3.5). They were amazing completion models and openai basically dumped them with no equal cost or equal performance replacement. Instead all their models are opaque with no ability to work in completion mode where one actually controls the text input to the model.
These days no company even has completion models where one controls the text input fully. Worthless.
same I just switched to OpenAI after using flash 2.5 lite for almost everything at our company. We spent thousands just to build this workflow now Google says screw off
It'd be interesting to know how much the Intelligence as a Service angle serves as a value-add in the minds of Google's executives.
You can get decent open-weight models now. That's not difficult. The difficulty is 1) running them and 2) compliance.
My company runs Claude on GCP's Vertex AI solution. We're in the US healthcare IT space, so the models need to be from somewhere that American healthcare agencies and companies have traditionally been okay with sourcing code from - which means the US, Canada, and maybe Europe. The stuff that handles PHI/PII must be in the US. The expense of hosting is more of a PITA than most customers want to go through this early in the technology's lifecycle, and intelligence gains are simply a matter of degree for most business tasks.
In theory, we could find some open-weight model (likely from China) for our development agentic work and host it anywhere you can host AI models. We don't, though, and I think Google, OpenAI/Microsoft, and Anthropic see that as the core of their business.
Is that statement based on token price? More and more it seems that $/token hides as much as it reveals. Token efficiency, tokenizer differences, etc. I'm not saying that you are wrong, I am just saying it is becoming a bit more difficult making statements like this without a bit more research.
No word about updating Jules, which is still stuck on 3.1 Pro. I get that it's probably niche but I've really appreciated basically being able to give directions to Jules on my phone, then reviewing and merging a GitHub PR fifteen minutes later. It's been great for getting some progress in on a few personal projects during my commute when I can't exactly pull out my laptop.
Shame. I'm on the $20/mo Gemini Pro plan because the 5tb of cloud storage and the youtube premium lite were good enough perks, and my coding complexity needs were light enough for me to overlook Claude or Codex. But Antigravity is working better than Jules and it's basically giving me a taste of what I'm missing and it's harder to justify not trying out the competitors.
I do not, however I am curious about Jules support. I didn't know if this was a dead project or not. Seemed really interesting but then I didn't see much development/announcements/discussions around it. Last update from their changelog was as you said 3.1-pro support in March.
Really, what's up with Gemini still not supporting connectors/MCPs/plugins/whatever-they're-called-this-month on web? It makes it a non-starter for any kind of serious use.
3.5 Flash-Lite seems available in US region, as was 3.5 Flash; but 3.6 Flash looks Global only so far when pinging. If Google employees are watching, will this issue go away?
Their naming scheme is confusing. Branding has never been Google's strong suit and their marketing copy is pretty bottom-of-the-barrel[0]. Anthropic has a pretty clear set of models but Gemini decided to rebrand their Flash as Flash Lite (and presumably the future will see a Flash Lite Mini, a Flash Lite Mini Nano and a Flash Lite Mini Nano 3B) which confuses the pricing to high hell.
This plus the Vertex, AI Studio, Gemini, Antigravity. It's honestly too confusing to use. I need to use Gemini just to decide on which platform and which model to consider.
0: Famous Kurian Tweet: "We're announcing Duet AI for Google Workspace will now be Gemini for Google Workspace. Consumers and organizations of all sizes can access Gemini across the Workspace apps they know and love. We're introducing a new offering called Gemini Business, which lets organizations use generative AI in Workspace at a lower price point than Gemini Enterprise, which replaces Duet AI for Workspace Enterprise."
Spent half an hour just now benchmarking it against my current 3.5 Flash pipeline excited only see it regressed slightly (0.1% - 0.2% at most, for feature extraction work)
Seems like this is mostly a cost play by Google, hoping this doesn't bring 3.5 Flash capabilities to an end of life, and that 3.6 catches up or gets better.
I have a side business selling custom fingerprint jewelry and I use gemini nano banana to clean up customer submitted fingerprint images. This was a step I used to do by hand at 10 - 15 minutes per image and nano banana is the first model that is able to do the task (it is astonishingly good at it). I can't wait to see what the next nano banana can do, hopefully its released soon.
In other good news "the model has been trained to minimize refusals for beneficial uses.".
Otherwise, this news feels like a tiny incremental improvement on Gemini Flash series to make it more efficient with token usage, subagent and cost. Nothing big.
Regarding their benchmark scores on CyberGym, I wonder why they didn't compare their 3.5 Flash Cyber model with Fable 5. I mean they included Mythos and GPT-Cyber, so why not Fable 5 too?
They also mentioned Gemini 3.5 Pro is in testing and its about to become available very soon. Another thing maybe worth discussing is the announcement of pre-training Gemini 4. Sadly, not much technical details to discuss on. Many comments in here seem to mostly be about how Google is behind the others, but honestly, is it really worth the investment to be #1 in Artifical Analysis every week?
Google seems to have anorexia when it comes to model intelligence. They have an internal hard constraint on price per token it seems, and they are trying to squeeze out intelligence with limited compute.
I wonder if there is something with their TPU cycles that makes them want to postpone training a new model. My guess is that they have been on the same base model for 6 months and they may have waited for the next gen TPUs to train Gemini 4, which greatly limits how much intelligence they can increase and forces them to do cost efficiency increases.
I often use Gemini free web chat because it's generally quite good at web search-related questions (apparently it has direct token-level access to the Google Search index) but I noticed in the last two weeks output quality of 3.5 Flash seriously degraded. Maybe they were switching over systems.
Google's Knowledge Graph is a massive advantage no other competitor has. I don't think they've fully utilized its full potential but I don't know if any other company could've built something like Scholar Labs
Wow - Google does not even bother to show benchmarks of these models compared to the frontier and Chinese labs - only against previous versions. I'm not surprised. Having worked there for years it was amazing just how inwardly looking the company is.
Google has not changed. Following two facts are like tautologies by now.
1. Their AI efforts are very fundamental research oriented. They are really good at it.
2. Their productization sucks. The end products gets little attention compared to competition. It can be canceled at any time. You should never build anything around Google only APIs, AI or not.
I don't know if it's a rendering error since it looks like your site renders the SVG instead of hosting a static image of it, but the 3.6 macbook looks like an abstract art piece lol, both ff and chrome desktop
Because AA Coding "Index" consists only of two benchmarks (Terminal-Bench v2.1, SciCode) and generally fails to be meaningfully representative of agentic coding capabilities.
DeepSWE and FrontierCode are more realistic if you read up on what they actually measure. But the most realistic is to try it yourself. Benchmarks can only vaguely represent typical usage, and how you judge the result. Giving the same real task you have to a few models will make you understand them better than chasing benchmarks.
Could be a good tradeoff for the flash model though. 3.5 -> 3.6 is a tiny bit cheaper and maybe faster?
artificialanalysis.ai has it going from 165 tps -> 304 tps. openrouter.ai needs more data but it has it going from ~100 tps -> ~150 tps, though at peak 3.5 has reached 156tps.
I have just tried to switch to 3.6 instead of 3.5 in antigravity and it seems to constantly spit "critical instruction: STOP CALLING TOOLS NOW. YOU MUST WAIT FOR WAKEUP. ". I think I will switch back to 3.5
Kind of excited about this. 3.5 Flash on Antigravity has surprised me recently on a hobby project. When given opportunity to plan, it can deliver on tasks that would take me a while on my own and generates responses at blazing speeds - compared to what I'm used to at work with Opus 4.8 (granted I don't use Opus 4.8 on my hobby projects so just anecdotal). While with Gemini CLI I would just watch it run in circles and run out of 5h allowance before anything useful is produced (or even approached).
It is 17% more token-efficient than 3.5 and performs significantly better in coding and tool usage benchmarks.
It is also cheaper than 3.5:
> This enhanced efficiency is also combined with a lower price than 3.5 Flash. At $1.50/1M input tokens and $7.50/1M output tokens, 3.6 Flash reduces the overall cost per agentic task, making agents more cost-effective to build and run.
Always happy to see new Gemini releases as IMO Antigravity Pro 16.67/mo plan (Annual) is still the best plan available and have been pretty happy with Antigravity IDE.
If it wasn't for Gemini/Antigravity I'd have to go with a Max Claude plan, as it stands now I can get by with just a Claude Pro plan to get Opus when I need it, whilst using Antigravity as my day-to-day workhorse.
Unfortunately Gemini Flash became too expensive to use as a general purpose model (i.e. for AI features in Apps), luckily there are plenty of cheaper Chinese models to fill that gap now.
Don't know why are they even pursuing Gemini. Just download the Kimi, call it Kimini and serve it on your GPU. Maybe then train next architecture based on this!
Feels like they released this to ride the wave of press of GPT-5.6, Kimi K3, and Qwen 3.8. Doesn't feel like Google has much substance with this post except a bump in version and tweaked their pricing.
Gemini 2.5 Flash-Lite has been my go to for cheap document processing at scale (especially with 50% off batch mode), but they are really boiling the frog with pricing increases with each version:
gemini-2.5-flash-lite: $0.10 input / $0.40 output
gemini-3.1-flash-lite: $0.25 input / $1.50 output
gemini-3.5-flash-lite: $0.30 input / $2.50 output (a 6.25x increase over 2.5!)
Now watch them deprecate Gemini 2.5 Flash-Lite in the coming months...
LLM reception is truly extreme, even worse than AAA game releases.
Ever frontier lab lived it at least once : missing the frontier by a few months triggers extremly negative reactions, then you take back the lead for 2 weeks, and the hype cycle repeats.
3.6 Flash scores exactly the same as 3.5 Flash on the Artificial Analysis index. Better on some tasks, worse on others. Mostly within what I'd consider the noise window. Looks pretty much indistinguishable from 3.5 Flash, at least on these benchmarks: https://artificialanalysis.ai/models/gemini-3-6-flash
For anyone wanting a faster overview: I ran the Gemini 3.6 Flash and 3.5 series release notes through NotebookLM and generated a short video summary. Link: https://www.youtube.com/watch?v=SUFBhvQ2tY4
IMO Gemini has the best free tier models/app for everyday use. Muse-Spark is perhaps just slightly better, but has none of the connectivity to my GApps (for things like “create a recipe in my Google Docs from this image”).
Plus they are probably running these things on every Google search so saving tokens is a huge win for them.
I'm a big fan of the Flash-Lite models. They're exceedingly fast and deliver great outputs for high volume use cases where you need to process requests at scale. Can't wait to try the newer version.
I asked Gemma 4 E2B, and if you use it as a reasoning model, it will give a better answer (that it doesn't know; it was also using the date information from the prompt.)
Because they don't have a lot of parameters to store general Wikipedia knowledge. They're small. Use big models that have high parameter capacity to store general information. Or build a harness around the small model that searches a knowledge base/internet.
Use the right tool for the job. It's like asking why a screwdriver isn't good at sawing wood, or calling C a terrible language because it's hard to make CRUD apps with it.
That's wildly ambitious pricing by Google. You can maybe get away with spicy pricing at the SOTA edge but at the lower tiers everything is a lot more price sensitive.
You need to compare cost per task buddy boy. Cost per token doesn't tell you much when you don't know how many tokens a model will use to accomplish a task
So 3.6 Flash is a somewhat of an admission that Google miscalculated by charging 3-5x for 3.5 Flash what it did for 3.0 Flash (3x input and output costs plus large token inefficiency changes) despite only modest improvements?
3.5 Flash Lite is only a hair cheaper than 3.0 Flash, but I think 3.0 Flash is a massively more capable model?
Proof-of-life release while they figure out how to have a competitive frontier model release. My hunch is they pushed too far in the "omni" model direction, that they made something so ungainly, it wasn't as good for normal tasks.
It does seem like their releases are getting closer together. I get the feeling they realized they were trying to roll out to their entire ecosystem and now they’re focusing more just directly on the AI model itself. I think give it a little time and they’ll start to be one of the competitors too.
They are working on coming up with a better code name. You know, something like ”Fable” or ”Sol”, gotta have one these days. Personally I think they should go with “Mafia”. How cool would that sound? 3.6 Mafia.
I have no skin in this game and this comment will be gray in a few minutes BUT a friendly reminder that these types of threads are astroturfed heavily by competitor labs and any info should be taken with a massive grain of salt.
Especially when Google owns 15% of anthropic and serves them compute. Double especially when your boss (Hassibis) is also an early investor in Anthropic. Hell his NW might be more Anthropic than Google.
Yep, agreed. They still are not releasing anything frontier-class (Gemini Pro) at this point. Feels to me that they keep getting scooped by others (e.g. Kimi 3) and then are retrenching.
tl;dr: 3.6 flash is a bit smarter than 3.5 flash, but also a bit more expensive.
My results [0] put Gemini 3.6 Flash at the top.
3.6 Flash high has same $1.5 input price as 3.5 Flash, but output is cheaper from $9.0 to $7.5.
Google said 3.6 Flash is more token efficient, but in my tests it's actually LESS token efficient[1] than 3.5 Flash, so despite the output price reduction, it still costs more.
the real product is the naming confusion we made along the way. Gemini 3.6 Flash, 3.5 Flash-Lite, 3.5 Flash Cyber — at this point even the model cards need a model to explain them
Models are expensive and low performance. On top of that they make you jump through hoops to even use these models without being throttled even for the weaker models. The only reason we are using them is credits. As soon as credits run out we are switching immediately.
quite a good model, the speed/price/quality ration is a new golden intersection for me, not sure if its as good as grok 4.5 but quite fast/capable model.
It feels like AI is going to be the end of Google. The post-Schmidt company culture cannot produce consistent, consumer-friendly products that any sane person would want to use consistently.
Well, you can use the google models from Pi. Go to aistudio.google.com and set up an API key. There is a free quota, it's relatively large for the Flash Lite models.
...and last time I looked the limits were more generous for Gemma 4 there, but they have been tightened a bit. That's how it goes, always changing.
Plugged 3.5 Flash Lite into an existing agent harness that was previously using 3.1 Flash Lite and this shit just does not work. It's not following instructions and is not producing the correct tool calls.
I remember back when Gemini looked like it was the best model that this comment section was full of confident predictions that Google had "won" and that no one would ever catch up with them again. The most embarassing part is that I kinda believed them.
Why do you care that much? Just say it. You're letting an imaginary score determine if you should express your suggestion for improvement. That's a bit wild.
I think it’s safe to say Google seems a bit out of the top AI competition now. The “cyber” stuff also starts to become laughable with open models providing the full power without the crap Anthropic, Google, OpenAI are trying to frontload on you(I.e you are not allowed to develop/review a login system, pay a special cyber operation team to do it for you). They really deserve to become irrelevant in the future of AI.
Google watches over the last few months a flat out assault on the Pareto curve from American and Chinese companies. Release after release pushing the boundaries of frontier intelligence and price/performance.
And the response from arguably the biggest AI research labs in the world by headcount is Flash 3.6.
What do you do when you are given essentially unlimited resources and still find yourself falling behind?
Gemini 3.5 flash is already a pretty good model. But, unfortunately, the primary way you can interact with it for coding is through Antigravity - which is actively developer hostile.
It doesn't matter how good the model is if you're (mostly) forced to use it in Antigravity - which turns any model into crap.
I use 3.5 flash 10x more than any other model, despite have access to all of them. If I'm going to play a slot machine, I'd rather get the pain over with quickly.
We're almost five years into the whole GenAI thing and we're still relying on these guys to spoonfeed us incremental updates.
It's time for them to start focusing on open-weight models and efficiency. Otherwise there's just a layer of marketing hype and "will it do this?" that has to be cut through for evaluation of each and every release cycle.
Models are getting easier and easier to create. The money, if there's any here, is in the harness the user interfaces with, and the data centers running them.
Google seems to be falling way behind the pack. antigravity cli is pure trash. gpt 3.5 pro is now behind and isn't released yet. GPT 6 and Fable 6 releasing next month. What the hell is going on over there ?
Google was late to coding agents and as-per-usual fucked it up with their crazy project management culture.
Usually Google gets away with it due to inertia, however this time they are paying a heavy price because they missed out on the training data that Anthropic and OpenAI have gathered with claude and codex.
> we have taken an intentional approach to deploying 3.5 Flash Cyber. The model will be exclusively available to governments and trusted partners
Screw your government! US and Israeli governments should get the least access, but of course we all know they'll be the (only) ones to get full unfiltered access.
With all the naysayers on Gemini models I'm curious how many people actually use Gemini regularly?
For me, Gemini models are the most usable. Claude Opus and Mistral always try to turn queries into one-shot enormous commits, which just burns tokens, time and annoys me for something which is still wrong more often than not.
Gemini seems far better at listening to instructions and giving me what I actually want, on top of using far fewer tokens and wasting my time. Fable is the only model that's come close to Gemini Pro for me.
And as this is about Flash, it's exciting, I find Flash can usually get the right answer pretty quickly and without too much nonsense.
I keep saying this and people dont believe me, but I have b2b saas systems with actual agents running around the clock, and the performance/stability of the flash model is higher than most other models.
Meaning, its predictable with tool calls, wont spin off a million tools/do weird behavior, its reasonable. Even sonnet in a real world decision making scenario is not reliable, or will reason so long its incredibly expensive.
The benchmarks arent catching all the value, and most people have never actually ran an ai agent in a real context that matters
Who's most people? What are you talking about? Most people here use agents every day and I wouldn't trust flash or pro to touch any important project of mine because they're both terrible compared to the competition, waste of time every time I give them a chance
not a google fanboy by any stretch... though i've been thrilled with the flash line of models... i exclusively use it on high, and have found it to be a great fit for increasing productivity 10-fold while maintaining quality... sure it can't just go off and one-shot a bunch of work, but at the complexity level i tend to work at, neither can the frontier in a robust way that i can be confident in... sure i have to be in the loop more, but that helps keep me grounded and course-correct earlier before wasting tokens... and when you sufficiently spec out a coding/software problem, and i mean really document all of the critical nuance, it will successfully satisfy the constraints... the quality is rarely acceptable on first-pass, but it forces me to stay connected to the architecture more than i would be if using a frontier model... i've found this to be a happy middle-ground of productivity and awareness...
tested the models on aistudio. despite that the knowledge cut off is march 2026 it still knows nothing about 2025!
you can check by asking "list notable world events in 2025, only list unplanned" on aistudio. or you can ask for Charlie Kirk, it also does not know. I tried it multiple time to ensure that I didn't not get routed to older models!
> but google has search
irrelevant, without deeper knowledge about cutting edge technologies or latest libraries, all of it suggestions are crap. even you ask it to search it will still use outdated keyword thus only getting outdated information.
Google desperately needs to make some leadership changes within their Gemini team now that they've been surpassed by 3-5 open weight models and risk loosing frontier status all together in the near future.
Open weight models aren't likely to be open weight in the long-term. China has started considering export controlling and limiting access to model weights [0].
They don't need to be open weight in the long term; once there's an open-weight Fable-level model with 1M context it'll be pretty much good enough for all coding tasks, no need for new models.
At this point, I think google should consider becoming a hyper scaler for anthropic and open ai, and I predict that that is exactly what they do. The model is no longer the most valuable part of the stack.