Microsoft Tries To Address Privacy Backlash Over New Windows 11 ‘Recall’ Feature
Back in May, Microsoft announced that it was bringing a new feature to Windows 11 dubbed “Recall.” According to Microsoft’s explanation of Recall, the “AI” powered technology was supposed to take screenshots of your activity every five seconds, giving you an “explorable timeline of your PC’s past,” that Microsoft’s AI-powered assistant, Copilot, can then help you peruse.
The idea is that you can use AI to help you dig through your computer use to remember past events (helping you find that restaurant your friend texted you about, or remember that story about cybernetic hamsters that so captivated you two weeks ago).
But it didn’t take long before privacy advocates understandably began expressing concerns that this not only provides Microsoft with an even more detailed way to monetized consumer privacy, it creates significant new privacy risks should that data be exposed:
“It makes your security very fragile,” as Dave Aitel, a former NSA hacker and founder of security firm Immunity, described it—more charitably than some others—to WIRED earlier this week. “Anyone who penetrates your computer for even a second can get your whole history. Which is not something people want.”
A lot of the detailed analysis on this illustrated that privacy most assuredly wasn’t anywhere near the forefront of Microsoft’s thinking as this was being developed.
Microsoft initially tried to calm user concerns by insisting all the processing happens on your local device and isn’t shared with Microsoft. Given the last decade or two of corporate privacy promises, that didn’t go over well. So Microsoft is now taking additional steps to try and address concerns, including making the new service opt-in only, and tethering access to encrypted Recall information to the PIN or biometric login restrictions of Windows Hello Enhanced Sign-in Security.
That this kind of stuff didn’t occur earlier to a company with the kind of money, staff, and resources of Microsoft perhaps says more than the company’s belated fixes do. Microsoft certainly isn’t the worst example in the AI space, but the obsession with making a quick buck from “AI” hype certainly has more than a few companies and employees forgoing basic fucking reason and due diligence.
Microsoft of course has a larger problem in that a lot of people really don’t like Windows 11 that much; or at least don’t see a reason why they should migrate from Windows 10. Microsoft is hoping to end “free” support for Windows 10 next year, but it remains the most popular Microsoft operating system by a pretty wide margin, something probably not helped by new feature privacy kerfuffles like this.
Microsoft’s Copilot+ PCs with Recall are slated to launch June 18.
Filed Under: ai, copilot, privacy, recall, security, windows 11
Companies: microsoft
Comments on “Microsoft Tries To Address Privacy Backlash Over New Windows 11 ‘Recall’ Feature”
It's much, MUCH worse than it appears
Attackers have already realized that they don’t have to invest time/effort in writing/debugging code to grab everything that happens on a Windows 11 system: Microsoft did that for them, and paid for it, and pre-installed it. All they have to do is leverage it.
Which includes enabling it if the user didn’t.
And they will, because this is a keystroke logger on steroids. It’s one of the worst pieces of malware imaginable, and it’s part of the OS.
And then things get worse. Because it’s only a matter of time until employers figure out that this is a way to do continuous surveillance on employees – in or out of office – and switch it on for every system they have. This has all kinds of consequences and all of them are bad.
(Let me enumerate one for you: some environments that handle sensitive data are under legal and contractual obligations to delete that data — which means all copies of it, which means backups and archives and well, all copies. How’s that going to work out?)
And then things get worse. Because every government on this planet is going to want a backdoor into this and will try to strongarm Microsoft into providing one as a condition of doing business in their country. Also has all kinds of consequences and all of them are bad.
The fact that this wasn’t shouted down in an internal meeting well before the first line of code was written is not just a red flag, but a bright flashing red flag with a siren next to it. Microsoft’s now trying to slap patches on it (PR, code, etc.) but apparently they’re not going to do thing they should do, which is to rip it out.
What's this?
Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »
Windows search
Given how bad Search is on Windows right now (it’s blind, and it tries to drive you online asap even if you want to find content in a file on your own machine), I’d not trust an AI enhanced version to do any better.
I’m so glad that I can opt out by simply not buying any of the CoPilot+ supporting hardware.
Re:
Where’s the money in directing you to a file on your own machine? Who’s gonna sponsor that?
Re:
Through leaks, people have managed to get this Recall feature running on normal x86 PCs without an NPU. And… AMD, nVidia, and Intel have announced NPU products covering a wide range of hardware price points and feature sets now.
The future is unfortunately now old man.
Re: Re:
1+2= The future is unfortunately bad Search on Windows
Re:
My view is simpler.
I just don’t trust a company that has a proven track record of shitting on their customers and anyone else they can get to hold still long enough, for PROFITS.
How in God’s name is a series of screenshots not the worst possible kind of search?
Re:
In Microsoft’s mind, I’m guessing it’s because of OCR.
Re: Re:
Obsessive Compulsive Recall?
Re: Re: Re:
Optical Character Recognition.
Recall regularity takes a screenshot of each opened window then use AI locally to do OCR and extract some structured text of each window content.
It’s to avoid asking devs to implement the whole tool on each application (which would require years to get it working) but it also mean that Recall is unable to recognize clear passwords and personal information from regular text.
And since all is stored in a file (secured by the greatest Windows security practices), it’s pretty much the dream for ransomware wannabes.
It seems that the only thing Recall can successfully filter out is copyrighted video, because you can mess with user data, not with lawyers.
Security based on a PIN?
4-8 digits isn’t going to cut it. Longer than that, you’re getting into password territory (which is good), but still subject to brute force attacks.
But either way, someone (say, law enforcement, or abusive spouses) what wants to snoop on your computer, will still have a field day with your Recall images. Hope that you never once have your (other) passwords and codes visible on the screen.
Windows 11 is objectively bad. There’s nothing in it that Windows 10 can’t do as good as or even better than it. UI is worse. Bloatware is worse. Privacy is worse. Security is about to become (even) worse. It’s unfortunate that I can’t do everything I need on Linux (yet). I’m considering paying for that extended support after they stop providing security updates to the wider audience.
I’m hoping Windows 11 goes the way of Windows 8 and 12 or whatever name they are going to give the next iteration gets things more right than wrong. I won’t hold my breath though.
Re:
That’s really the only line that matters to Microsoft. All that other stuff you’re bitching about is pretty standard; people do it with every new Windows release, ever since Product Activation was announced. Swear they’re gonna stop using Windows if it gets any worse. Or they’ll stick to old versions (a couple of university friends switched to Windows 2000 to avoid XP).
Things never get better. They just get normalized, so people think things aren’t as bad anymore. Maybe people come up with a few hacks to disable the worst of the worst.
“Extended support”, by the way, will only give you 3 extra years. Maybe you can get another 5 or so after that, if someone figures out how to make the embedded updates installable on non-embedded versions (as some XP die-hards evidently did from 2014 through 2019).
“all the processing happens on your local device and isn’t shared with Microsoft”
The processing happens on your machine they say, what about the storage?
Total Recall
Get your ass to Mars!!
Windows 11
I think the reason why Win11 isn’t popular is because nobody can install it on older PCs. Out of the 5 PCs I have in the house, I’ve got exactly one that meets MS’s standards for Win11.
I’d have installed it on at least two of them by now. I think that is skewing the numbers.
Re: Old PCs
Try using a virtual machine setup. Win11 as a guest on a Win10 host is not unheard of.
Re: Re:
Shit for gaming. Just use Ubuntu at that point.
Re: Re:
If you’re savvy enough to use VMs you might as well run a Linux distro instead of Windows 11.
Re:
In my CADCAM community, Win 11 is causing enough problems that people are actively “downgrading” from 11 back to 10. If you look up the market share, 11 is presently losing market share, while 10 is gaining.
Re:
I just bypassed the requirements and installed it on all the PCs at home, except the server. All seem to run fine, aside from occasional driver issues, but that’s nothing new since Windows 95. YMMV.
windows spyware 11
i like win 7 but have to downgrade to win11/10 because some of the programs i use wont run on win7 anymore! then when it time to update. it slows your shit down until you hit the update button! even with all the mircosucks crap minimalized. i still don’t like win11 spyware!
Just because someone figured out how to turn it on remotely and disable the indicator it was active is no reason to think this was a really fucking bad idea.
It's much, MUCH worse than it appears
Attackers have already realized that they don’t have to invest time/effort in writing/debugging code to grab everything that happens on a Windows 11 system: Microsoft did that for them, and paid for it, and pre-installed it. All they have to do is leverage it.
Which includes enabling it if the user didn’t.
And they will, because this is a keystroke logger on steroids. It’s one of the worst pieces of malware imaginable, and it’s part of the OS.
And then things get worse. Because it’s only a matter of time until employers figure out that this is a way to do continuous surveillance on employees – in or out of office – and switch it on for every system they have. This has all kinds of consequences and all of them are bad.
(Let me enumerate one for you: some environments that handle sensitive data are under legal and contractual obligations to delete that data — which means all copies of it, which means backups and archives and well, all copies. How’s that going to work out?)
And then things get worse. Because every government on this planet is going to want a backdoor into this and will try to strongarm Microsoft into providing one as a condition of doing business in their country. Also has all kinds of consequences and all of them are bad.
The fact that this wasn’t shouted down in an internal meeting well before the first line of code was written is not just a red flag, but a bright flashing red flag with a siren next to it. Microsoft’s now trying to slap patches on it (PR, code, etc.) but apparently they’re not going to do thing they should do, which is to rip it out.
Re:
E.U. courts will declare data-transfers to the U.S. illegal yet again, and then legislators will rush to re-legalize them?
There were all kinds of concerns raised regarding Windows 10’s non-optional telemetry collection, including crash-dump uploads that could include private data. How’d that work out? Everyone seems to have just forgotten about it. “I care about privacy, so I’m not gonna upgrade to Windows 11; I’ll stick with Windows 10.” (See, for example, the comment by “Ninja”. Such things happen with every major Windows release. When 12 comes out, people will vow not to upgrade from 11, because they like privacy.)
Re: Every government... is going to want a backdoor
This is what hit me. Anytime your computer was remotely involved in someone else’s crime, suddenly your entire computer activity history is prone to discovery.
But then your furry porn habit becomes a part of public record and that is the least of your problems.
Re: Re:
If porn habits were a big worry, people would have freaked out about browser history (a feature that’s apparently appreciated by a few people, but mostly goes unused). And they’d only be accessing porn sites over Tor.
Re: Re: Re: Porn Habits
Hence my use of least though kink-shaming is commonly used as a means of discrediting character in courts, and while furry porn is pretty innocent as fetishes go, our mainstream culture finds it conspicuously squicky.
That said, the real problem is once law enforcement or DAs decide you’re a bad guy (or need you to disappear into the penal system) any CFAA violation will do, and those are easier to come by than traffic violations.
I doubt that this Microsoft Windows 11 feature named Recall will be used in any high security area. Windows 11 will probably not be used there either.
Not a huge loss for MS I guess.
Re:
Ha, ha. Windows is frequently used for “high-security” stuff in the USA, such as classified military information. The military may be slow to upgrade and sometimes demand special concessions from Microsoft, but they will eventually upgrade.
I think there’s a good possibility Recall will be used in such places. Of course, they’ll convince MS to let them have a group policy setting that redirects the data to an alternate location, ostensibly to detect misuse of sensitive data. And possibly all the AI stuff will be disabled, too. But once people get used to the idea that every computer will spy on everything its users do, the type of organization that’s fond of “bossware” isn’t gonna be able to resist.
Re: Re:
“Windows is frequently used for “high-security” stuff in the USA”
How frequent is this at what level of security?
You may be correct that the rent-a-cops at the local pawn shop might be using a microsoft product.
“military may be slow to upgrade”
They may still be using windows 98, not very secure, just do not connect it to the internet.
“I think there’s a good possibility Recall will be used in such places. ”
Why do you think this? Specifically, what places?
Making copies of classified material of any sort in any form is strictly controlled … unless you are Donald and have aspirations of prison living.
Re: Re: Re:
To quote Ed Snowden’s book “Permanent Record” (available on Library Genesis):
That’s how Snowden got all those documents that were later leaked, which the US government claimed were extremely damaging to national security. The specific “level of security” was mostly “TOP SECRET”, according to document markings. SharePoint only runs on Windows, which is to say that Windows was used for all that stuff. (One might also notice that many of them were from Microsoft programs such as PowerPoint.)
Despite the name, there are secrecy levels above “TOP SECRET”, and I don’t know what operating systems are used for that.
Yes, it’s strictly controlled, and I suspect someone at the NSA is salivating at having that level of control. Had Snowden’s desktop been screenshotted every 10 seconds and sent to some NSA manager, the leaks could’ve been prevented—or so they may claim (nevermind that this manager could just as well be the next Snowden, leaking data via precisely these screenshots).
But that’s just, like, my opinion.
Re: Re: Re:2
“Had Snowden’s desktop been screenshotted every 10 seconds and sent to some NSA manager, the leaks could’ve been prevented”
I doubt that.
In addition, a requirement of very high security will forbid any and all connections to the outside world via any and all methods.
Re: Re: Re:3
So do I, but that doesn’t really matter. Do you doubt that some power-seeking asshole might try to claim that? Or that their boss might believe them?
Okay. So what? If Snowden’s accesses in Hawaii were being logged to Fort Meade, that wouldn’t count as “the outside world”. And those sites have to be connected, because Snowden’s job was to administer the documentation to be distributed across multiple NSA sites.
This feature, on the consumer side, will get people used to the idea that there may be a permanent record of everything shown on their computer screens. Whether or not such data, on their corporate/government systems, goes to Microsoft or someone else (local or remote) is a minor detail that will probably be adjustable via Group Policy.
Re: Re: Re:4
Ok, so what?
” get people used to the idea that there may be a permanent record of everything ”
And many cheer this on like it is the best thing since sliced bread.
Re: Re: Re:
So true. Some portions of the Army (at least) are still using Multiplan!!
Re:
Windows 11 is already in secure areas.
Source: I work with protected information.
Re: Re:
Protected information?
Do you allow the windows11 box to connect to the internet?
If so, it is not protected.
Re: Re: Re:
Probably “protected” enough to meet government requirements and keep 31Bob from being charged with espionage. And if you tell us that the government’s view might not match reality, well, welcome to reality.
Re: Re: Re:2
“Probably “protected” enough to meet government requirements ”
Facepalm
Which government requirement(s) are you referring to?
Probably is insufficient when it comes to things that really matter. For example, doors falling off aircraft may be a direct result of Quality Assurance levels in use being considered as – Probably sufficient.
Re: Re: Re:3
Evidence suggests that Windows is approved by the U.S. federal government to handle Classified information up to Top Secret (e.g. based on the NSA leaks). If 31Bob is talking about handling such data on a government-provided Windows system, your opinion about its security doesn’t much matter. It’s “good enough”, even if it’s not good.
Re: Re: Re:4
Yes, I have been aware that my opinion doesn’t much matter for some time now. I will continue to have opinions regardless.
‘It’s good enough’ reasoning has been identified as the root cause in many failures. I’m confident it will continue to cause failures unabated.
Re: Re: Re:5
Sure. I share your opinion, but what can we do about it? We can avoid using Windows and/or disable privacy-invasive features like this on our personal systems. But most people use banks, government services, hospitals, lawyers, and such, and they’re not gonna change because people say it’s not good enough. I once thought that maybe they’d change if people were relentlessly exploiting their systems, but the last decade’s disproven that. I also left a job in part because of the invasive spyware they wanted to install on my work laptop, but most of my co-workers just went along with it (which means that, when we suddenly began working from home, the employer was able and authorized to activate microphones and cameras in our living rooms and bedrooms).
Re: Re: Re:3 No door fell off
They wouldn’t pay for a door. Customer’s fault
This is what happens when they ask their AI is this a good idea and it says yes.
It says they’re just your average company in general. Companies could not give any less of a shit about attempting foresight when making “breakthrough technology.” It’s become the norm to just rush to market and only walk back poor decisions if there’s enough backlash.
So, am I a bad person for holding a lot of shares in Microsoft stock, and yet I’ve been using nothing but Ubuntu on my personal machines for going on 8 years now?
Well that's one way to keep people from 'upgrading'...
The idea that it took this blowing up in their face before they made ‘have everything you do on your computer recorded and able to be looked through’ op-in is beyond horrifying, and I would dearly hope was done in spite of strenuous objections by everyone in the company that actually does know how computers and security works.
This is a feature that is so toxic that it should never have been seriously considered, never mind actually implemented, and as such the only correct response should be to rip it out of the system entirely, not try to play a game of ‘okay but just in case you want the most privacy destroying feature imaginable, here’s the button to turn it on…’