Question about CPU microcode under libre-linux
A comment on another computer forum this morning has brought up an interesting question about CPU microcode under linux-libre:
@hotaru on the Phoronix forum writes - "while I'm generally in favor of the idea of running only free software, I've always found Linux-libre's stance on microcode updates a bit odd. not updating the microcode doesn't mean you're not running a binary microcode blob. it just means that you're running the outdated blob embedded in the UEFI firmware or the CPU itself that the CPU manufacturer actually admits is buggy and insecure."
I'm assuming that the point of libreboot is to liberate the board and the CPU, but I guess I don't know enough about CPU microcode to be certain. Does this user have a valid point, or are they just speaking in ignorance of libreboot?
> not updating the microcode doesn't mean you're not running a binary microcode blob. it just means that you're running the outdated blob embedded in the UEFI firmware or the CPU itself that the CPU manufacturer actually admits is buggy and insecure.
As I understand it, this is true. However, when the CPU manufacturer "admits" that the firmware is buggy, we have only their word that this is the real reason for the microcode update, that fixing the bug is the only thing that the microcode update does, or that the microcode update fixes the bug at all. If you allow microcode updates, you might get bug fixes, but you might also get new antifeatures or backdoors.
Since the microcode is non-free either way, it doesn't seem like there is a difference freedom-wise between allowing or rejecting the microcode updates. Security-wise, neither approach seems ideal, but I'm not sure which is worse.
> As I understand it, this is true. However, when the CPU manufacturer "admits" that the firmware is buggy, we have only their word that this is the real reason for the microcode update, that fixing the bug is the only thing that the microcode update does, or that the microcode update fixes the bug at all. If you allow microcode updates, you might get bug fixes, but you might also get new antifeatures or backdoors.
> Since the microcode is non-free either way, it doesn't seem like there is a difference freedom-wise between allowing or rejecting the microcode updates. Security-wise, neither approach seems ideal, but I'm not sure which is worse.
I guess this brings up 2 questions:
a) does Linux-libre "reject" CPU microcode updates? Or are those applied separately by the computer user?
b) does libreboot in any way negate the need for CPU microcode? i.e., if a T400 or an X60 or an X200 is librebooted, does it still need to use any manufacturer-supplied proprietary microcode for the CPU?
Thanks for your help in clarifying, chaosmonk.
We compared a "libre" distro with a "non-libre" one here:
https://trisquel.info/en/forum/libreshop-x200-sale#comment-140329
a) Linux-libre does not include binary blobs. Microcode updates are binary blobs. The microcode update needs to be loaded on each system boot.
b) You cannot negate the need for CPU microcode. The CPU cannot work without microcode (unless it is i486 or older).
https://en.wikipedia.org/wiki/Microcode
https://en.wikipedia.org/wiki/Intel_Microcode
> a) Linux-libre does not include binary blobs. Microcode updates are binary blobs. The microcode update needs to be loaded on each system boot.
> b) You cannot negate the need for CPU microcode. The CPU cannot work without microcode (unless it is i486 or older).
So if a Linux-libre user wants to keep their CPU microcode up to date, is there a preferred method of doing so? Or is it simply not done by Linux-libre users?
Sorry for all the questions. I do not run any librebooted systems, and I assumed CPU microcode was not something I would be able to do anything about one way or another. If I have a choice of whether to use older or newer microcode, I'd like to find out more about the positives and negatives and about how one would go about making changes.
> So if a Linux-libre user wants to keep their CPU microcode up to date, is there a preferred method of doing so?
https://downloadcenter.intel.com/download/28087/Linux-Processor-Microcode-Data-File?product=873
"The normal preferred method to apply microcode updates is using the system BIOS, but for a subset of Intel's processors this can be done at runtime using the operating system.
[...]
In situations when the BIOS update isn't available, early loading is the next best alternative to updating processor microcode. Microcode states are reset on a power reset, hence its required to be updated everytime during boot process."
> Or is it simply not done by Linux-libre users?
It's a binary, not FOSS.
> the positives and negatives and about how one would go about making changes
It depends what your goal is. "Freedom"-wise you are not any better or worse if you update using any of the methods because in both cases you cannot use the hardware without its proprietary stuff. Assuming that an update is more or less malicious without knowing what it actually does is not based on verifiable facts.
> we have only their word
If you don't trust a particular manufacturer - why buy its product in the first place?
> you might also get new antifeatures or backdoors.
Can you give an example of those introduced through CPU microcode? Personally I don't know of any. Microcode seems way too low level to allow a complex feature like a backdoor. But I may be wrong, so please explain if you know more.
BTW there is a way to check if the microcode update fixes the bug. For example running the spectre-meltdown-checker [1] script will show you.
I found something related to my question:
http://syssec.rub.de/media/emma/veroeffentlichungen/2017/08/16/usenix17-microcode.pdf
> Can you give an example of those introduced through CPU microcode? Personally I don't know of any. Microcode seems way too low level to allow a complex feature like a backdoor. But I may be wrong, so please explain if you know more.
You know more about this than I do. It seems that if a microcode update can fix a vulnerability of some severity at some level then it should also be capable of introducing a vulnerability of similar severity at a similar level, but I don't have a precedent for this in mind. The link[1] from your other post seems to show that such a thing is possible in theory, but that does not necessarily mean it has occurred in practice, and from section 8.1 of the paper this kind of attack does not sound very practical.
> BTW there is a way to check if the microcode update fixes the bug. For example running the spectre-meltdown-checker [1] script will show you.
Thanks for the correction.
The boot firmware (BIOS/UEFI) and/or the kernel usually apply microcode updates on boot. Both may do it: The BIOS/UEFI may apply one but the kernel's newer and has a newer version that it then applies. Anyway, keep that in mind...
One argument I've seen in favor of microcode updates is that (people claim) Intel updates the microcode in the CPUs, like that there are different versions of the same CPU model with different microcode versions. At least, that is what is *claimed* -- I'm not saying it's true -- and that applying the microcode updates is equivalent to if one had simply purchase one of those. Some of the "evidence" that people have pointed to is doing cat /proc/cpuinfo | grep -i microcode and different people seeing different numbers for microcode versions of the same processor model. But this argument seems to fall apart. Here's why:
At one point, someone came by the #linux-libre IRC channel to ask how to find out what microcode version was in their CPU because they were not seeing an entry when doing that "cat /proc/cpuinfo | grep -i microcode". They thought that Linux-libre was doing something special about microcode here. It turns out, it was not and that upstream Linux has this same thing: Reviewing the source code of the kernel shows that it (the kernel) also has a conditional to not print the microcode version if the processor gives a verion number of 0.
It turns out that this person was using libreboot and Linux-libre and so had no microcode updates applied (refer back to earlier to where these are normally applied by the boot firmware and/or the kernel.)
So it seems that Intel's processors return 0 when there is no microcode updates applied, making it impossible to find out what microcode version is "burned in" to the processor. More experimenting is needed here but perhaps -- just perhaps -- Intel doesn't actully ship the same processor models with different microcode burned in and that the only reason people are seeing different versions is because even though they have the same CPU model, they have different computer models (with different BIOS versions, which in turn apply different microcode updates depending on what the manufacturer last updated it for) and/or perhaps they have different kernel versions which supply different updates.
So we can't really rely on /proc/cpuinfo showing information about supposedly "burned in" microcode versions because it's showing the number after the update (from either the boot firmware and/or the kernel.) And if it really is true that, without updates, all CPUs return a microcode version of 0 then perhaps they all ship with the same microcode versions burned in -- and this that whole argument that applying updates is equivalent to buying a newer version of the same CPU model is false because all the ones of the same model are the same and you can't really get a newer version of the same CPU with newer microcode. And even if you could, how would you be able to tell since they return "0" when no updates are applied? It's a black box.
And then going back to what started it all: "not updating the microcode doesn't mean you're not running a binary microcode blob"
It's hard to tell whether they're running any software whatsoever. From a black box perspective (going back to always getting 0), all we can tell is that their working is indistinguishable from a pure hardware implementation, even if there's a possibility of override through software.
>all we can tell is that their working is indistinguishable from a pure hardware implementation
This is a lot of really great info. Sounds like CPU microcode has some urban myths built up around it. I'll watch the video too, thanks!
> that whole argument that applying updates is equivalent to buying a newer version of the same CPU model is false because all the ones of the same model are the same and you can't really get a newer version of the same CPU with newer microcode
You are wrong. Just like firmware can change dramatically the way a device behaves (which effectively makes it a different version of the device), so can microcode. One simple example: spectre/meltdown mitigations which slow down the CPU are practically making it behave like something else. Yes, it is still a black box but a different one as it responds differently to the input it receives.
> It's hard to tell whether they're running any software whatsoever.
Read above. Applying a microcode update changes the behavior of the black box, so it is measurable.
>spectre/meltdown mitigations which slow down the CPU are practically making it behave like something else
How are you certain those aren't just the effects of the kernel mitigations? I agree with jxself. Too many unknowns and too many urban myths.
What do you mean "just the effects"? It is part of the whole thing (speculative execution is faster).
>What do you mean "just the effects"?
I'm not of the belief that the Spectre and meltdown mitigations are microcode updates. Those are just kernel boot parameters.
Spectre and Meltdown are neither microcode updates, nor boot parameters. They are CPU vulnerabilities. Mitigations for them come through microcode updates and through kernel. The kernel boot parameters are instructions to the kernel, not mitigations per se and not microcode updates.
>Mitigations for them come through microcode updates and through kernel.
I've heard talk of there being possible microcode updates to mitigate spectre and meltdown, but I haven't seen them in practice. In fact, there are published benchmarks showing that you can completely undo the slowdown effects of mitigation with a single kernel boot parameter. If the mitigations were sent via microcode, it seems to me that means that CPU slowdown would be hard-coded, and would not be so easily reversed via a boot parameter.
On a "libre" system that denies microcode updates you won't be able to see them for sure.
On a "non-libre" system with applied mitigations (one way or another): undoing the slowdown due to the mitigation is possible if you disable the mitigation (e.g. through a boot kernel parameter). Then the CPU remains more vulnerable.
> If the mitigations were sent via microcode, it seems to me that means that CPU slowdown would be hard-coded, and would not be so easily reversed via a boot parameter.
The slowdown is actually the "natural" (hardware) speed of the CPU (roughly and figuratively speaking). It is the speculative execution that optimizes the work of the CPU making it more efficient (as if it were faster hardware). The mitigations reduce/rework those optimizations in order to improve security which comes at the cost of speed. So you could say that "the slowdown" the actual "hard-coded" speed of the CPU.
I think you didn't fully read what I was saying there or didn't fully grok it. Please check again; I'm not saying what you appear to think I'm saying.
>I think you didn't fully read what I was saying there or didn't fully grok it.
Are you replying to my comment or zigote's? Or both?
zigote's (#11). The forum should really make that clearer. :)
I think I read your answer incorrectly as well. So you are saying that there are plenty of microcode "updates" that get applied via BIOS updates and/or kernel updates. However, the idea that chips leaving the factory come with different microcode for the same chip is quite possibly a myth. Do I have that part right?
And I believe what you are saying is that regardless of whether CPU microcode updates are received via BIOS or kernel updates, there's no clear way to tell exactly what the microcode update is doing, or even if it is doing anything different at all. Am I right on that as well?
In that case, it would seem that zigote could be correct in the argument to me saying that spectre/meltdown mitigations are delivered via microcode update AND new kernel boot parameters. I said the mitigations are only delivered via new kernel boot parameters, but not via microcode update. However, from what you are saying there's just no way of telling what the microcode update is doing one way or the other. Especially given the fact that a CPU reverts to its non-mitigated behavior when given the 'mitigations=off' boot parameter.
In the end, I think the best advice for end users comes from the video that you linked to: don't do things with your computer that are insecure in the first place, and then you won't have to worry so much about whether black box microcode is protecting you.
"I think I read your answer incorrectly as well. So you are saying that there are plenty of microcode "updates" that get applied via BIOS updates and/or kernel updates. However, the idea that chips leaving the factory come with different microcode for the same chip is quite possibly a myth. Do I have that part right?"
Yes.
"And I believe what you are saying is that regardless of whether CPU microcode updates are received via BIOS or kernel updates, there's no clear way to tell exactly what the microcode update is doing, or even if it is doing anything different at all. Am I right on that as well?"
Mostly right. We can tell that at least some behaviors of the processor does indeed change, yes. But yes, being proprietary and secret and all that we don't have a way to fully understand all that it might be doing.
The part that I think zygote missed was at the end where I quoted that one person saying "not updating the microcode doesn't mean you're not running a binary microcode blob", which seems to be a common viewpoint: That the microcode "burned in" to the processor itself counts as a binary software blob, and sometimes continues that "if you're running a blob anyway you might as well keep it updated" and I was trying to point out that this isn't necessarily true.
For an example just look at history. I'll point to older versions of the DEC PDP-10 as an example. You have things like wire wrapped processors. Transistors. Those sorts of things. Clearly absolutely no software at this level here; pure hardware. And yet, the Incompatible Timesharing System (ITS) that ran on these were able to load changes that altered how the processor worked. This was done in order to add paging support back in the day, which DEC originally did not include support for. That leads to my last paragraph in my original post that (referring to these Intel black box CPUs) that "it's hard to tell whether they're running any software whatsoever." zigote's reply was the changes in the processor's behavior (from loading microcode updates) can be observed ("Applying a microcode update changes the behavior of the black box, so it is measurable") and so that is supposed to be an example of how I am wrong and there *must* be software inside the processor because it's behavior can change. But as I've explained in this example we've been able to alter processor behavior even in the years gone by before they started to even *be* microprogrammed in the first place. Obviously we don't use that kind of old technology from the PDP-10 days anymore but just because the processor's behavior *can* change, it doesn't *necessarily* have to mean that there is in fact software running inside the CPU (the whole "not updating the microcode doesn't mean you're not running a binary microcode blob" thing.) This is why I think zygote didn't fully grok what I was saying; because their reply was about something completely unrelated (changing processor behavior) which wasn't what I was talking about and seemed to miss the point. As I've explained doesn't have to mean there is software in there. All that we people can really tell (from looking at the processor as people outside of Intel) is that its workings are indistinguishable from what could also be accomplished with pure hardware implementation (look back to the PDP-10 to see what could be done), even if there's a possibility of overriding some parts of the hardware through software like they did back in the day with ITS. As you say there are many urban legends and so maybe some believe them too deeply in order to realize this this particular fine point.
But this also seems to be very much turning into a case of https://xkcd.com/386/ so perhaps I should stop.
"In the end, I think the best advice for end users comes from the video that you linked to: don't do things with your computer that are insecure in the first place, and then you won't have to worry so much about whether black box microcode is protecting you."
Absolutely! I completely agree with that. I don't apply microcode updates and have no fear of things like Spectre and Meltdown.
jxself,
You can program how a machine behaves by modifying the hardware (wire wrapping, re-soldering, etc.) or by modifying only the firmware (a special example of which is microcode). I am not familiar with PDP-10 and ITS or how exactly it is "able to load changes that altered how the processor worked", so that example and the insufficient information you give with it doesn't quite prove that there is no "software inside the [Intel] processor" (or that it can't be proved).
Here is another example: It is possible to use more than 4 GiB or RAM with a 32-bit CPU through a modification of the OS. Still that doesn't mean that the OS modifies the behavior of the CPU or its actual address range. It has nothing to do with how higher level CPU instructions are translated into lower level ones (what microcode's main functionality is).
Software, being instructions + data, is really the input which you provide to the hardware. To make it as simple as possible, let's look at simple AND circuit with 2 inputs (A and B):
A = 0, B = 0, output = 0
A = 1, B = 0, output = 0
A = 0, B = 1, output = 0
A = 1, B = 1, output = 1
The inputs A and B can either be pins on a hardware circuit or inputs to a software function. If we assume the hardware is hard (unchangeable) the results above will always be those. But if for some reason the output for the same combination of A and B is different (e.g. 1 AND 0 = 1) then surely this means that the hardware has changed (and is not hard but soft).
Even with the complexity of a real CPU this principle is observable and measurable, though not so simply. So I wonder what your point is by implying that there may be no (software) microcode running whatsoever. Are you suggesting that just because it is a binary/proprietary it does nothing? Or that it may do only harm again for the same reason? Or that the paper I shared about the reverse engineering of microcode is about reverse engineering something that doesn't work? I don't quite understand what you are trying to say.
"In the end, I think the best advice for end users comes from the video that you linked to: don't do things with your computer that are insecure in the first place, and then you won't have to worry so much about whether black box microcode is protecting you."
I don't see any such advice in that video. At which time position do you see it?
You have no way to evaluate fully whether a microcode update is insecure, so assuming that it is "insecure in the first place" is like assuming that there is a little devil (or god) living inside the CPU without having seen it for yourself.
What you *can* evaluate are the changes an ucode update claims to do (e.g. using spectre-meltdown-checker).
Consider also the ending words of a core Linux kernel developer who surely knows more than all of us about all these things:
http://kroah.com/log/blog/2018/01/19/meltdown-status-2/
"And yes, I need to go find a working microcode update to fix my laptop’s CPU so that it is not vulnerable against Spectre…"
Of course these vulnerabilities cannot be fixed completely without actual change of the hardware. Some of the newly discovered ones can't be fixed even partially.
>I don't see any such advice in that video. At which time position do you see it?
Time position? It was the entire point of the video. Watch it or don't, I'm not handing out a Cliff Notes version.
> It was the entire point of the video.
No, it wasn't. I explained in my previous reply.
It seems a lot of microcode update concerns stem from things like Spectre and Meltdown. If that's where anyone's concerns come from I urge you to watch this video:
https://media.libreplanet.org/u/libreplanet/m/who-s-afraid-of-spectre-and-meltdown/
The problem with that video is there is that the speaker contradicts himself and jumps from one thing to another, often quite superficially which leads to some wrong conclusions:
"If all of the programs running on your computer serve you, what is the problem if one of them can read information from the other? They are doing just what you want. Right?"
Wrong. Suppose I write a piece of FOSS which reads the memory of root. It is doing just what I want and that *is* the problem because it MUST NOT be possible to do that. So being "free" has nothing to do with security.
Also "Run for any purpose" does not mean you can run GIMP for the purpose of web browsing.
"You could have modified them so they pass information in some more efficient way but you don't have to do that. You can leave them as they are. It's OK if the programs serve you."
No, it is not. I should not be able to read the memory of another user. But I am (using my FOSS).
"But if they don't serve you, well then you should worry."
The question is how do you know that something serves you. There are many running processes even in a FOSS system. To a non-expert it is not obvious which "serve you", i.e. one should be worried all the time.
"Of course programs that serve you are free software. That is very definition of free software - program that is under your control."
But that is not quite the definition of free software. In any case my Meltdown FOSS program from above serves me and is under my control.
"Microcode has to be free because it also controls the behavior of your computer."
Does it? Surely it doesn't control my PSU and my DVD.
"Some people listen to this speech and think 'Oh, if I just run free software I'll be safe' - No, that's not what I am saying. Freedom is not enough for security, it's a requirement. You actually have to work hard to audit everything and make sure you can trust the softwares that you are running. But if you don't have freedom then you are already lost."
What if I write a simple proprietary program which just outputs "foo bar"? What exactly makes it "insecure" and how am I "lost"? And what has any of that to do with Spectre and Meltdown? He seems to imply that if microcode and everything else was FOSS there would be no CPU vulnerabilities but the two are different and unrelated things. IBM's Power CPUs are vulnerable too.
Bottom line is: "Who's afraid of Spectre and Meltdown?" is not quite about Spectre and Meltdown (or microcode).
"Suppose I write a piece of FOSS which reads the memory of root. It is doing just what I want and that *is* the problem because it MUST NOT be possible to do that."
Says who? Everything on my laptop is mine. Why should I not be able to read and write anything and everything that I want to? It's ***my*** system. The four freedoms say I should be able to do with what what I will; including modifying the software to completely remove any and all security if I wanted to. That would also include to write programs that uses Spectre and Meltdown as nothing more but a different method for interprocess communication on ***my*** system: https://en.wikipedia.org/wiki/Inter-process_communication
"What if I write a simple proprietary program which just outputs "foo bar"? What exactly makes it "insecure" and how am I "lost"? And what has any of that to do with Spectre and Meltdown? He seems to imply that if microcode and everything else was FOSS there would be no CPU vulnerabilities but the two are different and unrelated things. IBM's Power CPUs are vulnerable too."
Did you really not understand how free software is a necessary (but not sufficient) thing for security? Assuming that you understand this, his first advice is golden: Stop blindly running random proprietary programs that strangers (i.e. websites) send you. That alone helps to minimize one's risk/exposure to this.
> Says who?
Someone who knows what a multiuser system is.
> Everything on my laptop is mine. [...]
That reminds me of a friend who proudly says that he absolutely always works and does everything as root because he desires "full power and control" of "his computer", running X etc. :)
I wonder if you realize that this is not about you, ownership or about any single user on a single computer, processing data isolated from other systems.
Surely there is much more to computing than the thinking "this is my system, so I must be in control". Suppose you administer a server or a cluster running only FOSS, on which many people process *their* data and run *their* FOSS. Now imagine that one of those users runs the Meltdown FOSS (or Rowhammer, or another). How will you protect the other users (or root)? Don't you understand that the FOSS cannot solve this issue and that this is a *hardware* issue which even FOSS systems like IBM Power have?
> Did you really not understand how free software is a necessary (but not sufficient) thing for security?
Perhaps you have not read this [especially the last paragraph]:
> Assuming that you understand this, his first advice is golden: Stop blindly running random proprietary programs that strangers (i.e. websites) send you. That alone helps to minimize one's risk/exposure to this.
FOSS gives a chance for public control but that doesn't guarantee at all that anyone uses that chance. There are thousands of FOSS projects on GitHub for example. The % of those which people are interested in is not huge. The % of people who are developers and who would audit the code (and each new commit) is much smaller. Additionally there is no guarantee about the moral qualities or the motives of the developers or of the organization which may be supporting and influencing them.
So "that alone" does not help to minimize one's risk but much more is necessary. Also NSA is not behind every software company. Software is proprietary mainly because of the nature of the market (competition, capitalism etc.) and not in order to be malicious. There are even FOSS developers who create proprietary software, so they can survive. Example: Libraw is FOSS but FastRawViewer and RawDigger (which use libraw) are not. All 3 products are by the same developer. Of course we want everything to be FOSS but as it seems capitalism and FOSS ideology are incompatible.
That said: One should not run blindly anything (including FOSS). If it is FOSS - one should at least check the culture of the community which develops it, maybe also run some simple tests in a secured environment.
"one should not run blindly anything (including FOSS)"
And with this I think you're beginning to understand what he's talking about in the video.
You are attempting to demonstrate better understanding but from the whole discussion so far I see the opposite.
>maybe also run some simple tests in a secured environment.
It seems that preparing the secured environment is impossible for us average users in the first place, though.
The whole microcode discussion is confusing. Adding a bit of mine:
- regrettably we do not even know all microcode, as there are some undocumented "features" inside.
- some CPU's even have a tailoired Minix (3) OS inside > https://www.zdnet.com/article/minix-intels-hidden-in-chip-operating-system/
- DefCon 26 talk: Christopher Domas - The ring 0 facade: Awakening the processor's inner demons
Video in my Nextcloud: https://grns3.rocks/nextcloud/index.php/s/GRiADyWLt45x8yx
- adding more confusion: DefCon 26 talk: Matt King (ex Intel) - Micro-Renovator: Bringing Processor Firmware up to code > video (My Nextcloud) > https://grns3.rocks/nextcloud/index.php/s/Tw8b2GSKy4XJSWc
- related: DefCon 26 talk: Christopher Domas - Godmode Unlocked: Hardware backdoors in redacted x86 CPU's
Video in my Nextcloud: https://grns3.rocks/nextcloud/index.php/s/H32fb8Pfm4aGH6Q
- related: Defcon 26 talk: Shkatov and Michael - UEFI Exploitation for the masses
Video in my Nextcloud: https://grns3.rocks/nextcloud/index.php/s/b9JDYJ2sSaXLDND
Kindly note: i prefer running Nextcloud on my own servers for video sharing above linking to you tube
P.S.: do not use a production server to test stuff :)
I also have the feeling that the libre hardware supporters are all scattered. eg. You have Libreboot developer, and you have libreboot-friendly-coreboot developer. Then you have plenty of Free Hardware vendors using CrowdFunding, RISC-V, TALOS II, ...
It's also great to host your own "media" server.
I prefer my media without ads and with privacy, thus nextcloud (fork of owncloud) serves me to avoid "social media".
As for my libre support, i already looked at t400 to replace my vintage laptop :)