391 points by hn_acker 6 hours ago | 54 comments
everdrive 5 hours ago
The experts were correct. Azure is the biggest pile of shit I've ever had to work with. Everything feels evolutionary. In other words, a new product in azure is barely a product at all, but a small appendage which totally inherits a bunch of preexisting Azure "stuff." And all this preexisting stuff may not really make sense for the product, and it might inherit stuff that makes the product much worse. But, it doesn't matter. To even think about using the product, you need to learn way more about the larger Azure ecosystem than you ever bargained for, and of course deal with Microsoft products that do not really integrate well because the teams don't talk to each other. Log formats, conventions, everything will be different as you float around to different parts of Azure. Basic security concepts, such as a SIEM will be implemented in such strange ways that you wonder if Microsoft has any idea what a SIEM even is.
anonymsft318 4 hours ago
As a Microsoftie of more than a decade... Yeah, I see this.

We have an internal system called Cosmos[0] that does a great job of processing huge quantities of data very fast. And we sat on it for years while the rest of the industry moved to Spark and its derivatives. We finally released it as Azure Data Lake Analytics (ADLA) but did a shit job of supporting/promoting it.

We built Synapse, and it's garbage. We've now got Fabric which I guess is the new Synapse. I wouldn't really know because I probably have five different systems that I use that basically do large-scale data processing, and yet Fabric isn't one of them; who knows, maybe it will become the sixth?

We've had numerous internal systems for orchestrating jobs, and it wasn't until Azure Data Factory that we finally released something externally that we sort-of-kind-of-but-not-really use internally. (To be fair, some teams do use it internally, but we're not all rowing in the same direction.)

I regularly deal with multiple environments with different levels of isolation for security. I don't even know how it's all supposed to work -- I have my regular laptop and a secure workstation and three accounts that work on the two. Yet I have to do some privileged account escalation to activate these roles; when I'm done, there's no apparent way to end the activation early, so I just let it time out.

These things are but a fraction of the Azure offerings, but literally everything I have used in Azure makes me absolutely HATE working in the cloud. There's not a single bright side to it AFAICT. As best as I can tell, the only reason why Azure makes so much damn money is because Microsoft is huge and can leverage its size into growth. We're very much failing up here.

[0] https://www.microsoft.com/en-us/research/publication/big-dat...

beaviskhan 2 hours ago
> I probably have five different systems

This is the story of Microsoft - five different ways to do the thing, none of which do everything, and all of which are in various states of disrepair ranging from outright deprecation on up through feature-incomplete preview. Which one do you use? Who knows, but by the time you get everything moved over to that one and make allowances for all the stuff the one you chose doesn't support, there will be a new more logical choice for "that one" and you'll have to start over again. Wheee.

blks 1 hour ago
And now slap widespread vibe coding and PRs that reviewed by LLMs without anyone giving it a proper look.
anonymsft318 1 hour ago
We are now definitely doing a lot of that. My manager has been saying things like, "I don't even know how it works, but I used AI to build [thing], and I just sent it to a PR." He's very strong technically, but the mindset has absolutely shifted to, "move fast and break things, yoloooooo". It's frustrating to say the least.
BizarroLand 1 hour ago
And most of that is done on Macbooks by people that either can not or will not use Windows OS.
rdtsc 1 hour ago
Wait, is this true? I would have imagined unless it’s about porting software or testing it, everyone would be forced to use Windows.

If it is true, wonder what the proportion is then: 25%, 50%?

chokolad 52 minutes ago
It's not true. Source - me, MSFT for 25 years.
Spivak 1 hour ago
I don't mean this as a jab, but would you use Windows to develop software? Especially Windows that has AD teeth sunk into it where everything is "managed by your organization." It's just a thousand small cuts for seemingly no good reason.
gambiting 0 minutes ago
>>but would you use Windows to develop software?

I'm a c++ developer and I wouldn't use anything other than Windows to develop software, for one reason alone - Visual Studio is a fantastic tool that is better than any IDE I have ever tried it and imho it's the best product Microsoft makes. It just works and works well. And most console toolchains are only on Windows, so outside of iOS development I don't really have a choice.

BizarroLand 1 hour ago
No, but I also wouldn't let people who do not understand the soul of the OS to rewrite it.

If I were the microslop god for 6 weeks, I would force everyone to go to a boot camp and use Windows 7 for 4 of those weeks so they could see what made it so good.

No invasiveness, an OS that felt like yours. Just enough eye candy to not be distracting but to also feel like a clean modern system. Low system usage at idle. Calm, clean, and ready to roll when you clicked a button.

Windows is NEVER going to be MacOS, but the dev teams seem obsessed with macifying windows while also wedging that AI abomination copilot into every line of code, so windows is getting a tag team of rapid enshittification on top of already having been massively enshittified, and at least some portion of it is due to the people being paid to make it not understanding what it is supposed to be, the niche it held, and the reason for windows existence.

With no soul, windows has to go.

pram 3 hours ago
Ugh this sounds like when I worked at Oracle/OCI. Some environments required a VPN, some a jumpbox, and some required logging into a virtual desktop, and then logging into a jumpbox. Just thinking about it gives me PTSD
john_strinlai 3 hours ago
any sufficiently large organization that is around for a decade or two trends towards spaghetti-access
alistairSH 3 hours ago
Yup, same boat here (mid-size company).

All the corporate stuff is behind Okta, so that easy enough.

But all the dev/test systems are a mix of SSO, individual logins, etc. At least they're all behind the same VPN (except when they aren't, but that's less common).

And of course, if you're a cloud engineer (vs "normal" software engineer), you also have to deal with AWS access, which is a whole different can of worms.

benterix 3 hours ago
And yet, somehow AWS managed to get this right-ish. They evolved, learned by making mistakes, and created de-facto standards (like object storage protocol) on the way, while at the same time supporting decades-old services. And I'm sure they'll withstand the current AI craze.
jen20 2 hours ago
AWS had the benefit of not trying to retrofit IaaS on top of a (already bad) PaaS.
DANmode 2 hours ago
So the problem is the team size, not culture?
jonnycoder 3 hours ago
Their support team likes to sit on things for a while too. I'm on day 4 of waiting for Azure to approve my support request to increase Azure Batch vCPUs from default of 4 to 20 for ESv3 series. I signed up last week and converted to a paid account. I'm going to use Google Cloud Batch today instead.
decimalenough 48 minutes ago
So the internal Cosmos DB has nothing to do with Cosmos DB the Azure product, which was an unwieldy assemblage of a graph DB, a NoSQL DB, a time series DB and an RDBMS last time I looked at it, but seems to have morphed into a "vector DB for AI" according to today's marketing?

https://azure.microsoft.com/en-us/products/cosmos-db

gpderetta 24 minutes ago
Ah, I remember Cosmos and SCOPE from my time at MS ~15 years ago! It was actually pretty cool technology. So is it still around?
siva7 2 minutes ago
Did someone say Active Directory?
markus_zhang 4 hours ago
> Everything feels evolutionary.

That's total "normal" for Microsoft at least from 2018, the year I started working with some of their products (Power BI mostly). They adopted a development model that is early release, fast iteration, and users as testers. No wonder everything feels experimental until much later.

Back then I just couldn't use Power BI. But fast forward a few years, I think it got a lot better since maybe 2020. You just have to stick with it for a few years.

dietr1ch 4 hours ago
> You just have to stick with it for a few years.

So, you have to be a paying tester? Incredible that MS can keep enough businesses as hostage to be able to operate like that.

markus_zhang 4 hours ago
Most of the time it's just part of the bundle. If you are heavy into SQL Server, Office 365 and Power BI then there is a BIG chance you are going to use Azure for whatever the reason.

People who take Azure up without previous MS product experience...not sure about those.

thewebguyd 1 hour ago
There's a few, mostly retailers who don't want to give money to Amazon as a direct competitor, for them Microsoft/Azure is more of a neutral party, and most businesses already use Microsoft in at least some fashion so already have staff internally familiar with MS products (as opposed to say, going to GCP instead).

For everyone else, it's like you said. "Eh, we are already knee deep in the Microsoft stack, why would we pick anything else?"

3 hours ago
calvinmorrison 4 hours ago
a LOT of stuff comes for free or marginal (10-100$ a month) so yes, you do pay but it's already 'baked into' the contracts people generally carry with microsoft, or something for IT to worry about when the yearly renewals show up
snapetom 3 hours ago
I worked at a hospital in that timeframe and they rolled out Teams. Up until they, shadow IT teams were running Slack just fine.

Man, what a horrendous pile of crap Teams was back then. The Slack teams were griping that they should just buy Slack, but Teams was the "enterprise solution." The problems were amplified during remote COVID work. Teams is fine now, but how many corporations went through years of frustration just because some IT decision maker said "Teams. Because it's enterprise."

markus_zhang 3 hours ago
Yeah that's the thing. Management who made the deals are never put into that frustration, or very rarely, and I always wonder, at least for the big corporations, if there is any greasy palms...
DANmode 1 hour ago
Manager humans will sell out your workflow,

and indeed your entire workplace,

for as little as a steak dinner.

markus_zhang 31 minutes ago
Man, at least make a few dinners…
bigbuppo 3 hours ago
Teams is still a horrendous pile of crap. It's just that you've gotten used to the stench. It has few redeeming qualities other than, "we don't have to pay for another subscription" and that's not even the case in the EU.
benterix 3 hours ago
Yeah but today you can at least have a video call more or less normally. Back then it was a hiccup after a hiccup, it was impossible to work normally, and yet orgs pushed it down everybody's throats as it was bundled.
snapetom 1 hour ago
Definitely. Besides the performance issues, back then, Teams barely had any features. One example was that it wouldn't show you who was talking. First time we had a call was with 30 people and I remember a manager calling out a director responsible for this decision jokingly saying, "and you don't know who I am because Team doesn't show you who's talking."

The UI is an overengineered mess and I'd rather use literally anything else, but to say it's still unusable is disingenuous.

TYPE_FASTER 4 hours ago
> You just have to stick with it for a few years.

Also see: SharePoint

DrewADesign 4 hours ago
Absolute contempt for their users at every level. It’s so transparent. This is the end game of anticompetitive practices for decades— they just don’t have to try anymore… for now. Some day they’ll either have to compete in good faith or sink. I doubt that will happen soon, but someday.
pluc 3 hours ago
It's hard to argue against contempt but... I'm gonna try. It feels like at the end of the line it's just a checkbox someone gets without having to consider the consequences of the changes. Either it's too big or there's too many levels where decisions get made and handed down to drones (or AI), but the people who decide seem to have no concept of what their products are used for and the people who implement features seem to have accepted that the system is so big that they can't understand all the impacts of their changes and have to rely on trusting commands from above - who may expect them to challenge from the POV of users or question things but never do. Anyway, this feels like what happens when managerial overhead and marketing KPIs smash into a complex product ecosystem. It all smells of IBM to be honest
DrewADesign 9 minutes ago
I don’t perceive benign neglect when they disregard UX for a product they’ve positioned so people essentially have to pay for and use it, while force-feeding them features they actively and vocally hate. Treating your customers as cash cows is fundamentally contemptful.
Spooky23 3 hours ago
Microsoft was always afraid of being IBM. They are more IBM than IBM.

When they started flying people in the beg that I buy 100 Surface Laptops, that was the confirmation of everything I had been thinking. All I could think of was IBM flying a dude from Italy in to talk for 15 minutes about their version of TeamViewer back in the day. We ended up talking about shoes.

dralley 1 hour ago
It's a shame. In the late 2010s there was a lot of hope for Satya Nadella, but it seems like the organization has regressed back to the mean.
BizarroLand 1 hour ago
Which is sad because the CEO's job is not to focus on the individual body parts but to make sure that the whole system is strong, beautiful, and healthy.

They can afford people who would do better. Windows 11 is trash. Azure is trash. Onedrive is trash. Outlook is trashier than it has ever been before, but it's not quite trash yet. Word is trash. Excel is rapidly enshittifying. Copilot is hot flaming radioactive tar cancer.

Does microslop even have a single thing left that isn't either completely terrible or worse than it used to be a mere 5 years ago?

Spooky23 16 minutes ago
> Outlook is trashier than it has ever been before

Which one? There’s two now! Lol

BizarroLand 8 minutes ago
Both. "New" outlook doesn't work with all of the add-ons and plug ins that "classic" outlook did. Both new and classic have copilot wedged into them. Classic has unasked for and unwanted Linkedin integrations that have to be turned off on a per-user basis, and it is patently clear that microslop has every intention of abandoning classic outlook the instant they believe that they can do so without severely alienating their userbase.
lokar 3 hours ago
If not contempt, at least disregard or indifference
jeffrallen 2 hours ago
They have to compete in good faith for developers, which is why VS Code does not suck.

But yes, normal Office users, where the company pays the bills, pay the price.

DrewADesign 13 minutes ago
I agree that VSC is solid for web dev or other script language workflows, and VS is fine, if a bit heavy-handed. That said, Windows native development is a freaking mess. Try figuring out what their recommended native UI kit is these days. Everything is half-assed and half-supported at best. Unless it’s going to either feed them a ton of marketing telemetry or let them bump up their supposed copilot adoption statistics, you’re yesterday’s news to MS.
HeWhoLurksLate 14 minutes ago
VS Code is also open source and forkable, the Windows kernel or Azure tech stack not so much
reactordev 4 hours ago
Azure is the color of the face you have after Microsoft beats you with your own wallet. They don’t want to give you access to anything, they want to own it and make you pay for it.
andy_ppp 4 hours ago
I’ve seen this in other “follow the leader” businesses too, they are not looking to even have working features, just parity on a spreadsheet with the market leader… I’m looking at you Gitlab.
cj 3 hours ago
I sometimes wonder if I would feel the same about AWS if I hadn’t already invested a significant amount of time learning the entire ecosystem, nomenclatures, patterns/best practices, etc.
manphone 3 hours ago
As someone who has worked with all three in many capacities, as is the worst by a mile. Don’t get me wrong. They are all very bad, but Azure is the king of shit.
lokar 3 hours ago
And the same applies to regions. Try running is most of the regions, each is a bit different. And its not historical / sequential differences, just random.
bmurphy1976 4 hours ago
How is this different than Amazon? Same problem there. Oh, you're using this new service? Need to view the logs? Want a nice friendly UI to do that? Fuck you here's Cloudwatch. Good luck.

Just to be clear, I'm responding to the parent comment not the article.

klooney 4 hours ago
I love https://github.com/lucagrulla/cw , it's like tail for cloudwatch. It's super fast.
bmurphy1976 4 hours ago
That's great but that's not really the problem. The real problem is Amazon likes to release services that depend on other services, but leave the integration work to us.

I'm convinced Amazon has many teams crapping out new features but they don't have the political clout (or manpower) to create a comprehensive product. They are mandated by management to use existing services, and thus we the users suffer because we have to manage all this extra crap and noise just to enable basic functionality.

It's maddening. And then also it's maddening to see another service from a different team that was able to throw off these shackles and actually make a product that is self contained. You get a taste of how good things could be, and then you're thrown right back into the IAM/SQS/Cloudwatch/Cloudformation/Policy/everything else under the sun soup.

debarshri 4 hours ago
Amazon is selling servers and storage. If you need to see logs properly, then get a right tool for it. Cloudwatch is a stop gap solution.
bmurphy1976 4 hours ago
See my other comment. Logs are just one small symptom of a larger problem of poorly integrated very complex services where the complexity is pushed onto the users and not properly managed by Amazon. Which sounds very much like the problems with Azure.
lokar 3 hours ago
My general approach is to only use the most basic services from each cloud. VMs, networks, L3 load balancers, blob storage, etc

Build the rest yourself. In many cases their higher level service is just the same open source package you would run, just managed worse.

r_lee 1 hour ago
this. with Kubernetes, you can get very far with just this and you won't have to deal with lock in BS either
jbombadil 6 hours ago
> [...]And because federal agencies were allowed to deploy the product during the review, GCC High spread across the government as well as the defense industry. By late 2024, FedRAMP reviewers concluded that they had little choice but to authorize the technology — not because their questions had been answered or their review was complete, but largely on the grounds that Microsoft’s product was already being used across Washington.

This sounds like the crux of the issue. The combination of: "tool can be used during analysis" and "analysis takes long" shifts the barrier of rejection from "is this tool safe?" to "is this tool so unsafe that we're willing to start a fight with a lot of other government agencies to remove it, find an alternative, etc?".

Not criticizing FedRAMP. Proper security review takes time. And probably more when dealing with vendors.

chii 5 hours ago
It's why these enterprise vendors want foot in the door at all costs.

They know that if they get entrenched first, it's impossible to migrate away. That's basically free money from a customer that has zero cost ceiling.

andychase 4 hours ago
That's false that Government agencies have 0 cost ceiling. Maybe DoD does, but most offices have extremely tight budgets.
kipchak 4 hours ago
As far as I know numbers aren't reported, but there's probably at least as many DIB GCC-H customers as government, who in part use it because the government does and it's compliant. Once they're locked in it's very hard to migrate.
debarshri 5 hours ago
Recently tried using Entra ID. There are 12 ways to enforce MFA, 20 days ways to disable users, 4 ways to authenticate users, Add conditional access stuff with 50 variables and templates etc.

You can customize the way you want. After configuring it, my colleagues could not log in. Thats one way to secure your organization.

mastax 5 hours ago
Out of all the SSO login flows Microsoft has to have the buggiest. It’s the only one I can remember routinely having issues with. Why are there so many redirects? And why doesn’t the “remember me” checkbox ever work?
CDSlice 4 hours ago
It is also the only SSO flow I have ever seen that fundamentally cannot work if you have more than one account remembered on your device. So far the only way I’ve found to get it to let you log out of account A and then log into account B is to clear all cookies otherwise it gives you permission denied errors. Have no idea how it can be this horrible
rgblambda 3 hours ago
Would container tabs solve that? They're pitched as helping separate work and personal logins.
SoftTalker 3 hours ago
I just run completely separate browser profiles to separate work and personal stuff. And I still sometimes need private mode or a throwaway profile to get some random thing to work.
throwway120385 4 hours ago
Yeah I have had this experience too. Woe betide ye if your company gets bought by another company with pre-existing Azure AD.
rendaw 2 hours ago
And then sometimes the "switch user" prompt doesn't work but it automatically logs you in with the wrong account to a system that account doesn't have access to, then drops you in a non-interactive "you're not authorized" screen. You have to find a working page, log out, then go back and try logging in...
genthree 4 hours ago
I haven't seen it in a while (perhaps mostly because I'm in Google stuff way less than I used to be) but for years multiple Google sites would get in a state where its auth would route me through about twenty redirects in a loop and never actually finish authenticating me. Clearing cookies and re-logging-in from scratch was the only fix.

Youtube was always involved, somehow, for some reason, even when what I was doing wasn't connected to Youtube at all or the account I was using had never even been intentionally used with Youtube. It'd route me through a few Youtube domain names.

(Microsoft's is indeed even worse, on some of theirs [Azure Devops, looking at you] I can't use them in pinned tabs because somehow they manage to get into a totally broken state where the page won't load due to whatever's happening with their auth flow in the background, and no method of reloading the tab fixes it, and it does this every couple days—but copy-pasting the same URL to a new tab does work)

asdff 17 minutes ago
It is still like this? I remember it being terrible trying to log into xbox.com 15 years ago.
bombcar 5 hours ago
I've always assumed the billions of redirects are setting cookies so all the various systems "work" but I have given up trying to understand it.
pc86 27 minutes ago
Why, 20% when logging in, do I actually get logged out? I'm sorry if I was already logged in, why the hell are you asking me to log in again?

Having Microsoft on your resume is a huge red flag.

yoyohello13 5 hours ago
That’s Microsoft. 1000s of features and none of them really work the way they are supposed to.
ploxiln 3 hours ago
it's "Enterprise" grade software! need to check the boxes for the procurement process (actually working is a separate department)
yoyohello13 1 hour ago
Exactly! I can’t even count the number of times we’ve been in the discovery phase of a project and see “Oh this MS product does that! Cool”. Then when we get to the actual implementation realize it’s a broken mess. It’s sales driven software development, they just need to get you far enough along to sign the contract, then it’s too late to back out.
b0rgedhuman 4 hours ago
[dead]
joezydeco 5 hours ago
There are extra ways to do that, but they're on a document deep in a Sharepoint directory that you can't access.
debarshri 5 hours ago
Moments like this, I miss clippy.
3 hours ago
Pxtl 4 hours ago
The problem is modern MS doing three contradictory things at the same time:

- FB's move fast and break things. Constantly launching new libs.

- Linus's we do not break user space. Great commitment to backwards compatibility.

- Never deprecating dead products until they've been de facto abandoned for like decades.

This combination means every MS product is a labyrinth of overlapping APIs with no guidance as to which one is actually the good one. Some are abandoned garbage, some are brand new and incomplete, and some are both, and there's no way of knowing which are which even experts can mislead you.

0cf8612b2e1e 3 hours ago
Well said. It feels like Microsoft is willing to release the intern’s poorly thought out product, and then commits to support the garbage design for all time.

Microsoft, you are a behemoth. There are few domains where you actually compete. Give your products a minute to breath before you cast them in stone.

DANmode 55 minutes ago
> no guidance as to which one is actually the good one.

To some extent, you’re/we’re the ones deciding that,

because there’s entirely different teams heading the separate offerings,

and none of them are going to offer a potential footgun like:

“hey, we’re not the best modern path into xyz type projects, check with our colleagues on the Blazor team”,

unless someone makes them.

b0rgedhuman 4 hours ago
[dead]
jjtheblunt 5 hours ago
same experience for us, and then they email the living shit out of you about how your weekly entra id stats are good or bad, and you can not opt out of these emails.
lostlogin 5 hours ago
> they email the living shit out of you

This sounds like LinkedIn.

debarshri 5 hours ago
Wait a minute. It is owned by Microsoft.
lostlogin 4 hours ago
It’s a relentless horror. I signed my wife up to track down a driver that crashed into her.

I think LinkedIn spam is worse than being in a crash.

hedora 5 hours ago
Same here, except with Minecraft and XBox One.

I don’t understand how they have non-zero market share.

ryandrake 5 hours ago
I remember trying to buy $9 worth of Minecraft In-app Whatever for my kid, and the goose chase Microsoft put me on just to log in and buy something was totally out of this world. I ended up needing to contact their fraud department around step 74.
asdff 15 minutes ago
Wow I had no clue they even had in app crap for minecraft. Got to put the kid on the java build.
doubled112 5 hours ago
I'm still annoyed that I can't share those Minecraft purchases with a family.
alexpotato 5 hours ago
For Minecraft they inherited a gigantic userbase from Mojang and then made it 10x harder to add new users.
genthree 4 hours ago
I did it for my kids to have accounts and I do not understand how anyone who hasn't built a Gentoo from Stage 1 has a prayer of managing to buy Minecraft Java Edition for their kid, and making it actually work.

Then you've got the hell of overlapping permissions systems on the console and the Microsoft account, to get any amount of online play working on a console if you also get Bedrock. On the Playstation, especially, the error messages also love to not tell you which of the two systems is blocking you, so you get to guess. And Microsoft's site for managing those permissions is so confusingly-laid-out that even after doing it three times in a row I still felt lost on it.

I never did solve the problem of getting Minecraft Java Edition to run on a kid's MacBook with allowlist-only Web access. It wants to contact ten or so apparently-randomly-selected-from-an-enormous-pool IP addresses on every launch. I never did find documentation of which IP blocks I needed to allow, and couldn't guess at it from the IPs themselves. If they'd just used domain names... I must have manually hit "allow" a bunch of times during twenty separate launches, and it was still presenting me the same number of prompts every time, because there was no overlap in the IPs contacted (adding insult to injury is that I'm sure all but at-most two of these were spyware horse-shit that had no actual generously-necessary role in running the software, but it'd fail if it couldn't reach them)

asdff 14 minutes ago
I was supposed to have a license through my alpha build purchase but microsoft made it impossible to transfer over. So now I just see it as my right to pirate the game until the end of time.
DeathArrow 2 hours ago
I ripped Entra ID from one of our projects and replaced it with Keycloak.
sysops9x 4 minutes ago
The vendor lock-in angle is the real story. Once you have AD, Exchange, Teams, and Azure all tangled together, the exit cost is enormous. I've watched orgs get approval for a full cloud migration based on cost savings projections that completely ignored the engineering cost of the decade-long migration that follows. The security issues are real but the procurement inertia is what keeps them renewing.
iscoelho 5 hours ago
Microsoft has never been good at security, and that is why their centralization to cloud is absolutely terrifying.

I'm reminded of Storm-0558 [1] where a stolen signing key was able to forge authentication tokens for any MSA / Azure AD / Government AD user. They downplayed the severity. Just imagine if that level of access was used to pull a Stryker on a nation-wide scale. That is an economic disaster waiting to happen.

[1] https://www.microsoft.com/en-us/security/blog/2023/07/14/ana...

Rygian 5 hours ago
I'll do you one better: stealing the signing key was not even necessary.

https://www.bleepingcomputer.com/news/security/microsoft-ent...

iscoelho 4 hours ago
I knew there was another incident that I was forgetting, insanity... I don't understand how Microsoft keeps getting away with this and everyone just forgets.
someguyiguess 2 hours ago
When people's income depends on them forgetting... they tend to become amnesiacs.
natas 3 hours ago
because time to market is more important than security (at microsoft)
notepad0x90 3 hours ago
Oh please, that could happen at any company. Humans screw up.
iscoelho 1 hour ago
But it doesn't. Full authentication bypass exploits are extremely rare and unheard of among tech giants. Maybe account takeover/recovery, sure, but full bypass? It just never happens.

Microsoft goes beyond that: they've managed to have a critical vulnerability in almost every authentication product they have ever created. It's exceptional.

ovidev 5 hours ago
The Justice Department CIO who pressured FedRAMP to approve GCC High was hired by Microsoft the next year. I wonder if this shouldn't invalidate the authorization in the first place?
gertrunde 5 hours ago
It's not very clear from the article, but I get the feeling from the context that the 'pile of shit' quote referenced the package of documentation about the service rather than the service itself.

(That seems to be the main complaint, that Microsoft never provided the clear information required to conduct the assessment properly).

charles_f 3 hours ago
> The tech giant’s “lack of proper detailed security documentation” left reviewers with a “lack of confidence in assessing the system’s overall security posture,” according to an internal government report reviewed by ProPublica. > > Or, as one member of the team put it: “The package is a pile of shit.”

Yes, it seems pretty clear from that quote that the reviewer said the security package was a `pile of shit`, and propublica went on to extend that to the cloud itself. Not that I want to comment on the merits of Azure's security, but that sounds pretty clickbaity from propublica to me. A more appropriate title would have been

> Federal Cyber Experts Thought Microsoft’s Cloud Security documentation Was “a Pile of Shit.”

evan_a_a 3 hours ago
MS was (and still is it seems) unable to produce the data flow diagrams that FedRAMP wanted, ones that other cloud providers had no problem with. If the documentation is in such dire state, then the system itself is likely to also be in a dire state. I.e. The documentation is a pile of shit, so the system is also a pile of shit.
4 hours ago
21asdffdsa12 5 hours ago
Wait- so they basically threw up their hands? No documentation! Not evaluable? Thus clearly of value for somebody? Big stamp, job well done! NEXT?
mrguyorama 4 hours ago
Yes. US bureaucracy regularly gets told "You have to have <thing>" but because it's against a lot of people's ideology, they aren't allowed to build it internally or develop any sort of actual expertise for such a thing, so their only choice is to buy whatever is offered no matter how bad it is.

For example, our state government says "We will do X Y and Z which all require data science expertise, but we did not approve the $60k a year Data Science position, so instead we are forced to hire a Data Science contractor for $120k a year, and they can't really be fired, and they are terrible at their job"

And then people wonder why things suck all the time.

A lot of state's buy their Obamacare marketplace service from a company I am familiar with. That company is entirely incompetent. They cannot follow basic instructions. They cannot triage a bug at all. They do not read freaking tickets. They take weeks to respond to an issue. They cause bugs regularly in ways that imply they don't have functional source control. They continually fuck up basic feature requests. They change the service in ways that contravene the literal law. The law that was comprehensively explained to them by people I know.

But they can't be fired, because the state is legally compelled to provide this service, and is not really allowed to hire a few engineers to build it in house. They could go to a different software contractor, but all the options are just as bad because it's an entirely captured market.

Obama started a "Digital Services" group in the federal government to actually build systems internally and develop expertise to mitigate some of this, and they built stuff like tax filing solutions for free for Americans. So Trump killed it and hollowed out it's corpse for DOGE.

blizdiddy 4 hours ago
Emergency notifications are done the same way! Its communism to fucking build it, so let’s have a team of a few engineers make an API to control government infrastructure from incompetent contractors on AWS, offer no real means of testing, breaking changes, downtime… and folks wonder why Hawaii is told bombs are coming
crawdog 21 minutes ago
Entropy is real. Microsoft has lessened the friction of purchasing vs their competitors. Public Sector this may be their only choice because they set the groundwork decades ago to do business with most organizations. It may not be the best solution, but it offers the least resistance to getting something up and running.
GeoSys 17 minutes ago
So which is the most consistent cloud offering, that doesn't seem like a bunch of disparate services loosely wired together? GCP? AWS?
exabrial 6 hours ago
I'm guessing the requirements were written in a way that only Microsoft's cloud could with the bid.

Thats why you have Windows in the Pentagon instead of something secure.

gertrunde 5 hours ago
The sheer amount of conflict of interest with folk involved in this later getting employed by Microsoft is a bit crazy.
flir 5 hours ago
There was definitely a point (late 90s?) when Microsoft finally figured out how to play the game. Coincided with the antitrust stuff.
scottyah 4 hours ago
To be fair, it's not always out of maliciousness. A lot of gov workers/contractors join the supplier company because they know the product and how to fix it better than the people currently at the company. Similar to the guy who infamously got hired at Apple just to fix a bug.

You're just forced to use vendors and if you actually care about the mission, it's just a different team on the same mission.

Of course you know you're being taken advantage of, and long-term maybe you should have gone to the non-technical side to fight it, but at the end of the day you just want to keep the young boys being shipped off to war safe, and you're much better suited to achieve that by remaining on the technical side.

...or so I've heard.

markstos 5 hours ago
Frustrating that FedRAMP is both a pain to get compliant with and also apparently is not a strong signal of actual security.
colechristensen 5 hours ago
I see you've never worked in a compliance environment before.
Havoc 5 hours ago
And may such evil days never come to past
NoSalt 1 hour ago
EVERYTHING about the federal government contracts program sucks ass! In the beginning, it was good as you didn't want people forcing through their brother, mother, 2nd cousin, next door neighbor, Satya Nadella and their "company" as a contractor without oversight cough Kristi Noem cough. However, it has devolved into a mess. The entire thing needs to be scrapped and re-engineered.
thayne 1 hour ago
I don't have much experience with Azure but I was amazed at how many things in AWS GovCloud don't meet FedRAMP encryption requirements. For example, none of the lambda runtimes have FIPS certified encryption libraries available, and you have to bring you own, which is rather complicated to do.
caseysoftware 5 hours ago
Was this approval before or after evaluators discovered this?

> Microsoft on Friday revised its practices to ensure that engineers in China no longer provide technical support to U.S. defense clients using the company’s cloud services.

Ref: https://www.cnbc.com/2025/07/18/microsoft-china-digital-esco...

kqgnkqgn 3 hours ago
This is my opinion only, I'm sure some have had different experiences - but:

Azure's success as a cloud provider is mostly a result of their sales team and having an existing relationship with non-technical leadership. "We already pay them for Office and Exchange, let's just buy this new 'cloud' thing from them too".

Azure is barely considered an option at all within tech companies, yet is surprisingly widely adopted by non-technical companies that don't know any better (ie, that don't have a technical / engineering voice or representation within leadership).

AWS = Likely technically the best, for now. Mostly unreasonable pricing, and less motivation to seriously negotiate given they are the 'default' cloud provider for most of the industry. Kind of feels like they have peaked though, and are slipping more recently. Inevitable, or bad leadership changes?

OCI = New-comer, attractive pricing and hungry for business. Might be able to avoid mistakes other providers have made? Reliability struggles though. Parent company has a bad reputation in some circles - but probably not with decision makers. Making huge (unwise?) investments - that will either come crashing down in 5 years, or seriously pay off. Layoffs, but going for massive growth...huh?

GCP = Notably different underlying technical choices than other providers. Folks are maybe a bit less pragmatic, and more academic. This helps them in unique services (Spanner?) but hurts in most other areas. They've matured, and are btwn AWS and OCI in reliability. They are probably not as hungry for business as they should be given how far behind they are.

klooney 4 hours ago
> Potential Conflict of Interest: The government relies, in part, on third-party firms to vet cloud technology, but those firms are hired and paid by the company being assessed.

Hah. First time looking at FedRAMP?

The real reason for this, of course, is accounting, it moves it off of the government's books.

jakubadamw 5 hours ago
Little has changed since Bill Gates tried to install Movie Maker.
yoyohello13 5 hours ago
Basically exactly what my org did. The momentum of being a Microsoft shop is hard to fight against.
gurjeet 3 hours ago
> These highlights were written by the reporters and editors who worked on this story.

It's unfortunate that people have to claim the authenticity, rather than the users of AI having to disclose use of AI/LLM. I wish it was the other way around.

robtherobber 6 hours ago
Wow, Microsoft is really pushing the wrong boundaries in every direction, isn't it? Executives must be thinking, like many before them, that Microsoft is too big to fail.
joe_mamba 6 hours ago
Executives only react to share price movements. If share prices are high because whatever investors think, then execs will just open another champagne bottle.

Steve Jobs was the last tech CEO who didn't care about wall street and only care about quality products and consumers saying that if customers are happy, then the share price will take care of itself. But most companies are share price first, customer later.

shrubble 4 hours ago
This fits perfectly with traditional Microsoft strategies of getting a foot in the door and then having the users’ internal pressure on the organization to help get the Microsoft product established.

Decades ago, Lotus 1-2-3 on top of MSDOS was the lever; today it’s GCC High.

brudgers 5 hours ago
Given the scale and scope of the Federal Government. what are the alternatives to Microsoft?

Building in house.

Outsourcing to consultants.

realo 4 hours ago
IBM? Redhat?
nonameiguess 4 hours ago
I think there's some context missing here. For those who don't remember, the CIA back in like 2014 or so built out private data centers with classified versions of AWS services and all IC workloads that don't require specialized hardware was supposed to be using. DOD historically used it as well for classified cloud workloads, but wanted its own, and this was the JEDI contract, which was also supposed to go to Amazon, until Trump got into a fight with Jeff Bezos in 2019, canceled the contract, and awarded it to Microsoft instead. Amazon sued, and Biden decided to just award the contract to everyone and split it between all the major cloud vendors. That still doesn't mean anyone can actually use it without FedRAMP approval, but well, there you go.

The alternative was AWS, which has been operating at every classification level for over a decade at this point. It's now split between Amazon, Microsoft, Oracle, and Google, which is especially amusing because Google withdrew from the original bid process when they were still pretending to give a shit that their employees don't like working for the military.

sam-cop-vimes 4 hours ago
Microsoft has been selling piles of shit since the beginning of time. The fact that they keep selling is the biggest triumph of sales/marketing over decent engineering.
hn_acker 6 hours ago
The original title is:

> Federal Cyber Experts Thought Microsoft’s Cloud Was “a Pile of Shit.” They Approved It Anyway.

dogleash 5 hours ago
> By late 2024, FedRAMP reviewers concluded that they had little choice but to authorize the technology — not because their questions had been answered or their review was complete, but largely on the grounds that Microsoft’s product was already being used across Washington.

The article talks a lot about conflicts of interest, but this is the line I went looking for. A bureaucracy fighting itself over goal prioritization, and what's a necessary roadblock vs red tape is the less sexy but more meaningful problem at the core of this.

Once the government decided they wanted the product, they were going to find a patsy.

fdghrtbrt 5 hours ago
If you "went looking for" this line, you're just reading into the statements your preconceptions.

I on the other hand have no expectation, and so it's not clear whether the "bureaucracy fighting itself" is a cause or a symptom. You're implying it's a cause and the solution is "less red tape". But it could be just a symptom of conflicts of interest, and less red tape just leads to more efficient corruption.

Again, you're just reading into it what you already believe in.

4 hours ago
Eridrus 6 hours ago
I think plenty of software is a pile of shit and still derive value from it.
snovymgodym 5 hours ago
Yeah I'd go so far as to say that most useful software is "bad" in some way.
Y_Y 1 hour ago
Worse is Better
mock-possum 5 hours ago
Exactly, better the pile of shit you know than the pile of shit you don’t know - or the pile of shit that is u knowable.
iamleppert 5 hours ago
Azure is easily the most expensive, least reliable and worst cloud available. It's borderline scam. An example today, I provisioned high IOPS SSDs (supposedly) and what is actually connected to the instance? A spinning hard drive! I didn't even know they were still made, but I guess Azure uses them and scams their users into thinking you're getting an SSD for $700/mo when its really an old hard drive.

I would warn anyone far and wide to avoid Azure at all costs, especially if you are a startup. And especially if you are doing any kind of AI because the only GPUs they have available are ancient and also crazy over-priced.

If I cared more, I'd try to migrate away from Azure. But I don't, and that's probably Azure's business model at this point.

otterley 5 hours ago
I’d love to see proof of your claim that they provisioned a hard disk when you requested an SSD, or, at the very least, tests that showed that the IOPS you requested were not delivered. Can you show us the receipts?
stackskipton 2 hours ago
Azure using SRE, I call BS. You don’t see underlying storage, it’s mounted as either SCSI or NVMe device as one HD. It’s obviously backed by massive fleet of drives just like EBS.
kevincloudsec 4 hours ago
the product got deployed across the government while the security review was still in progress. then fedramp approved it because it was already everywhere. seem like i saw a lobbyist or two with a broom sweeping something under a rug...
stainablesteel 40 minutes ago
when someone says they work at meta, they get weird looks, but no one assumes they're incompetent

when someone says they work at microsoft, they get weird looks, and people assume they're incompetent

rukuu001 1 hour ago
Just like everyone else tasked with buying Microsoft
skywhopper 4 hours ago
Azure is bad. But to be fair, every security summary of IT services I’ve ever read — or written! — for over 25 years has also been a “pile of shit”. It seems to be inherent to the cybersecurity game that everything is judged based on meaningless check boxes and nonsensical explanations. Meanwhile the actual security posture is obscured and ignored.
scottyah 4 hours ago
Staying afloat on cyber compliance takes so much time and energy there's no room for actual cybersecurity analysis.
gffrd 3 hours ago
A pile of shit you have leverage over is better than a pile of diamonds you don’t.
franktankbank 3 hours ago
Microsoft is great at greasing palms
riffic 2 hours ago
all clouds are.
SanjayMehta 4 hours ago
A rigged RFP, and some very happy lobbyists, chortling into their single malt all the way to the bank.
FrustratedMonky 5 hours ago
Is this just a case of MS needing to merge a lot of platforms, and there are gaps and overlaps.?

Maybe the critical question, are they making continuing improvements? Especially to merge conflicting functions.

Like when they bought Minecraft, or Skype. Each already had user management. Xbox was a mess. Merging them all took a lot of years.

j45 5 hours ago
Maybe the gaps are a frature or benefit at the same time.
babypuncher 3 hours ago
okay what the hell is a "cyber expert"?
alexjplant 1 hour ago
Although "cyber" is a prefix used to denote a relationship to high technology there exist people who use it as a stand-alone term to mean "cybersecurity", e.g. "I work in cyber." It's very confusing given that it came from the word "cybernetic", the hundreds of other words that begin with it as well, and the existence of the term "cybersec" which is unambiguous and only a bit longer.
pissedoffadmin 5 hours ago
I fucking hate microsoft, i'm so sick of this retarded fucking bullshit
dwa3592 5 hours ago
Exactly, and that is the moat- a pile of shit that everyone can smell from afar.
fredgrott 5 hours ago
its as funny as the IA research reports from DORA dev which all seem to be sponsored AI provider ads instead....
Arubis 3 hours ago
I mean, they also bought the F-35.
mystraline 3 hours ago
Yeah, but this is how things work at that level.

Microsoft can be abhorrent. They will always get the contracts. Why? Corporate welfare.

Microsoft will drive the rules. Why? Too big to fail.

Microsoft will push their slop. Why? Cause they have contractors after contractors in the federal government pushing MS solutions. Doesnt matter if they're bad.

And, who'd pay for a 3PAO audit of a Linux distro? Ubuntu and Redhat have. Its a $120k moat.

DeathArrow 2 hours ago
Is there a big cloud platform that isn't a pile of shit?
bigfatkitten 1 hour ago
[dead]
natas 1 hour ago
[dead]
robutsume 4 hours ago
[dead]
wsesamemr81 2 hours ago
[dead]
Heer_J 3 hours ago
[dead]
pissedoffadmin 5 hours ago
[flagged]
ddtaylor 5 hours ago
The government does most things poorly and with little regard to budget or quality. They can't solve problems that are much simpler than cloud computing, so why should I expect them to perform better at a more complex problem?
ContDestroyer 4 minutes ago
"Things government is great at:

-Running public schools -Running public transport -Running public utilities -Running public radio and TV -Running Amtrak -Running Fannie Mae/Freddie Mac -Running it own budget -Running the military -Running Heatlhcare.gov -Running the Postal Service -Running Social Security -Running Air Traffic Control

...and this is just in the United States. You should see countries where everything (or nearly everything) is government-run. I've been trying to immigrate from the US to Cuba, Venezuela, Laos, China, or North Korea for decades! It's absolutely paradise to be in a place where evil private corporations are not in control. It's been shown time and again that the government is the best at solving problems.

Just imagine how great Germany would have been if all of it would have been taken over by East Germany?

Sure, government run everything is not always perfect, but it's a heck of a lot better than the opposite."

...SAID NO ONE. EVER.

Hizonner 5 hours ago
Sure. Your average private corporation would do much better at sanely evaluating Microsoft's cloud, and sanely acting on that evaluation.

Right.

You bet.

Absolutely.

ContDestroyer 31 minutes ago
Nope neither the public entity nor the private corporation... it should be the market. WE decide with our business who is worthy, or not. Nice try, comrade!
Hizonner 21 minutes ago
Fascinating.

Enlighten me further. How exactly will "the market" decide where the government, or a corporation, or even an individual, chooses to buy computing services? I'm very stupid, so you're going to have to explain step by step exactly how "the market" will do this. I mean, here I thought that choices like that were the inputs to the market.

Let's do it for the corporations first. I'm Microsoft. I need the market to decide for me where I should buy motherboards for my cloud data centers. Where do I apply to get "the market" to tell me that?

whoknowsidont 5 hours ago
I think this perspective has resolutely been debunked at this point.

The government has historically, routinely, consistently, solved problems more complex than cloud computing.

The only way you'd think otherwise is if you had some other motivation to pretend otherwise... some sort of ideology.

cptskippy 1 hour ago
> The government does most things poorly and with little regard to budget or quality.

That's a common line by conservatives who are actively sabotaging government with policies and laws which they then point to as evidence of such inefficiencies.

hiddencost 5 hours ago
Basically false. They're better at health care. Better at education. Better at feeding people. Better at charity.
MrBuddyCasino 5 hours ago
Theres no need to be THIS cynical.
hrmtst93837 2 hours ago
[dead]
notepad0x90 3 hours ago
Suddenly everyone on HN is an expert on Azure infrastructure.

it isn't the best but it's really great at a lot of things feature-wise. top-notch documentation as well (despite what these "experts" said).

Most companies literally run on Azure these days. Persistent hackers will get into any network, that's a guarantee, that's APT 101. It's law of averages. If it truly is "a pile of shit" given how it is probably the most used cloud platform by the most customers, including governments, and endless plethora of features and services it offers, shouldn't there be more compromises? 2-3 in a decade is hardly above what you expect for law of averages right?

Screw ups happen, but if it is systemic, you can't use one instance as evidence, you must establish a pattern of mishaps.

lokar 2 hours ago
I ran a one of the largest multi-cloud service across azure, aws and gcp.

Azure was hands down, obvious to everyone involved the worst technically. In capabilities, bugs/correctness, availability and support.

2 hours ago