> Why can't I have fast, low instruction cost communication between any two CPUs, regardless of which core it's on? In other words, why do I have to do that bookkeeping to have super fast communication between threads/actors?
L1 cache is literally inside of the core of modern chips. You can't beat its speed because its physically the closest speed possible.
Moving cache further away (ex: L2 or L3) means more wires, more capacitance, and slower electricity. Electricity doesn't move instantly: it has to fight the capacitance of wires. The wider the wires, the more electrons you need to put onto the wire before its voltage changes. There's also inductance: the longer the wire is, the longer you need to hold a voltage before electrons actually decide to move.
Second: L1 caches do a LOT of book-keeping for you. All cores on modern systems (anything more modern than DEC Alpha from the 90s) will have cache coherence automatically.
That's downright huge, and modern multithreaded programming would be damn near impossible without the ordering guarantees of basic cache coherence. (yes: even ARM / MIPS and their "relaxed memory model" are more coherent than chips from earlier eras).
So first of all, be gracious about the level of memory ordering available to the modern programmer.
------------
So how are caches coherent? Most modern caches use some form of MESI to communicate with each other (invisibly: this is "below" assembly language and is happening at the hardware level).
When one cache claims "exclusive access" to the data, no other cache will write to it (although they may read stale versions of that data from their own cache). When that core / L1 cache is done, it will "invalidate" the data.
Modern cache coherence protocols can go faster: MESIF for Intel means that writes may be (F)orwarded to other caches automatically (which is slightly faster than (E)xclusive -> (I)nvalid -> (E)xclusive in the 2nd core).
Again, this is all done automatically, because programmers want to have their data look the same from all cores (for some definition of "looks the same").
This does NOT mean that the data is in the correct order however (!!). L1 caches can change the order of reads and writes before the data is shipped to other cores. Indeed, this is the very POINT of L1 cache. If one core is doing a while(true) iter++; loop, its more efficient to merge the writes together and "batch" them out slowly. Ex: Core#2 may see "1, 10, 33, 600" each time it reads the value of "iter".
Indeed, the most efficient implementation of this while(true) iter++; loop is to have "iter" be a register private to a core that has absolutely no communication to the outside world at all.
Again, this is all done automatically, because programmers want to have their data look the same from all cores (for some definition of "looks the same").
In terms of bookkeeping examples, we have to make sure we're not inadvertently causing False Sharing.
This sort of thing is quite implicit as opposed to being explicit. This is because communicating with a caching+coherence mechanism is a bit of a kludge.
When one cache claims "exclusive access" to the data, no other cache will write to it (although they may read stale versions of that data from their own cache).
Get your head out of CPU caches and low level stuff for a moment. Imagine that someone proposes a high level communications protocol to you which acts like this. That would be ugly. That's the sort of mechanism you'd expect to see at a very low level of the networking stack. That's precisely the sort of thing a programmer wants abstracted away from them, so they can think of communication as communication, not as a series of details about the behavior of hardware.
> This is because communicating with a caching+coherence mechanism is a bit of a kludge.
Hmm. Right.
> Imagine that someone proposes a high level communications protocol to you which acts like this
It is very clearly not a gleaming tower of high-level abstraction but a hillock of filth with a little goblin upon it that actually scampers about doing the work. It sacrifices beauty so it can run fucking fast. Reality is gross, get used to it.
> Get your head out of CPU caches and low level stuff for a moment
I suggest you go and look at dragontamer's previous posts before you make rather insulting comments like that - he knows more about this area than you or I put together ever will. I read his & BeeOnRope's stuff (<https://news.ycombinator.com/threads?id=BeeOnRope>), you should too.
It sacrifices beauty so it can run fucking fast. Reality is gross, get used to it.
Again, it sounds as if you're trolling. Also, pretty much the same hardware that supports cache coherency would also support conceptually cleaner models for communication. A special register one CPU could write to, and the rest could read from, for example. (async propagation, of course) Using a cache line to for this purpose means that the entire cache line can't be used for other purposes, or at the programmer's peril.
before you make rather insulting comments like that
Not at all meant as a insult. Changing levels of abstraction is very useful, often powerful, and no one is above a reminder to do so.
Also, I've been reviewing your comments. They're somewhat quick to talk down and jump quickly to high handed conclusions about where other commenters are coming from
Fizzics, specifically a mixture of C and some electrical crap about signal propagation which I don't unnerstand. Einstein, he say no.
Please review the thread and try to find the understood references to Hennesy & Patterson which precede your linking to it. While you're at it, please substantiate where any mechanism I've suggested violates the rules of physics, C, relativity, or electromagnetism.
> A special register one CPU could write to, and the rest could read from, for example
Finally, an actual suggestion! OK, how are you going to mediate the thread access so one 'owns' it exculsively for writing while others can rad it? And when will the other threads know when there is something to read? And how wide is this register - 64 bits? And how much overhead will there be for this negotiation between threads? Something like:
x this register is mine for writing, all acknowledge please
x each thread acknowledges (taking various times as each thread lives on different cores/sockets so limited to the speed of the slowest)
x writer thread writes to register
x writer thread releases lock and signals one or more reader threads
x reader threads read and acknowledge lock so another writer can claim the lock
Quite an overhead. Now do that multiple times and it gets slow because it's only 64 bits, which is a lot of overhead for each chunk of data.
Compare that with multiple independent cache lines of 512 bits which can be filled and read independently, then passed around using existing, highly tuned and extremely fast cache coherence protocols that already exist.
> Changing levels of abstraction is very useful, often powerful, and no one is above a reminder to do so.
AFAICS you didn't specify anything until this post. How can we evaluate a claim of 'there has to be something better' until you give it?
> Please review the thread and try to find the understood references to Hennesy & Patterson which precede your linking to it.
Can't see any.
> While you're at it, please substantiate where any mechanism I've suggested violates the rules of physics, C, relativity, or electromagnetism
I was responding to this details-free request: "Shouldn't we have better hardware support for communicating threads?" Better has cost. You just seem unaware of the tradeoffs.
> Also, I've been reviewing your comments. They're somewhat quick to talk down and jump quickly to high handed conclusions about where other commenters are coming from
Yeah, not one of my finer features :( but if someone suggests something solid I'll talk, if someone is willing to learn I'll try to help, and if I'm wrong I'll admit it straight away.
This is evidence that either you're not familiar enough with these issues, so you need to have some things spelled out for you, or you have a rather narrow view of how things should be done at the level of machine language. Funny, because the entire discussion has been about hardware that doesn't commonly exist.
OK, how are you going to mediate the thread access so one 'owns' it exculsively for writing while others can rad it? And when will the other threads know when there is something to read? And how wide is this register - 64 bits? And how much overhead will there be for this negotiation between threads?
Absolutely none of that is necessary for pub/sub semantics between actors. (Either in the form of a register or a scratchpad.) Interesting that you jumped to those several unnecessary conclusions, then ran with it. Left as an exercise. (Hint, it's trivial to show you can't do it with just one instance. However, you can make do with just one tool.)
AFAICS you didn't specify anything until this post.
Sorry, but that has nothing to do with your consistently inappropriate tone, your lack of substantiation on my other points, and your Dunning-Kruger penchant for running off without understanding where the other commenters are coming from. I long for the USENET days when I had a 'killfile' for this stuff. Have a good day. I think I've exhausted the entertainment potential from this one.
'Scusi, I said "they [each core] share the L1/L2 cache". Badly put but I thought on x86 each core had strictly private L1/L2, and L3 was shared across the socket. Am I wrong in thinking L2 is private to each core?
I think L1 / L2 is private per core, but I'm not 100% sure. I "retreated" by stating L1 cache, because I'm 100% sure that l1 cache is private per core.
I know L3 is external to the core, and L1 is private. L2 is that iffy spot in my memory that I'd have to read a few manuals to remember exactly where it is... lol.
Not the parent, but yes on modern x86 L1/L2 is private. The key difference though is that the L1 is smaller and actually a physical part of the core, whereas the L2 is bigger and sits next to the core (read: further away from the execution engine).
L1 cache is literally inside of the core of modern chips. You can't beat its speed because its physically the closest speed possible.
Moving cache further away (ex: L2 or L3) means more wires, more capacitance, and slower electricity. Electricity doesn't move instantly: it has to fight the capacitance of wires. The wider the wires, the more electrons you need to put onto the wire before its voltage changes. There's also inductance: the longer the wire is, the longer you need to hold a voltage before electrons actually decide to move.
Second: L1 caches do a LOT of book-keeping for you. All cores on modern systems (anything more modern than DEC Alpha from the 90s) will have cache coherence automatically.
That's downright huge, and modern multithreaded programming would be damn near impossible without the ordering guarantees of basic cache coherence. (yes: even ARM / MIPS and their "relaxed memory model" are more coherent than chips from earlier eras).
So first of all, be gracious about the level of memory ordering available to the modern programmer.
------------
So how are caches coherent? Most modern caches use some form of MESI to communicate with each other (invisibly: this is "below" assembly language and is happening at the hardware level).
When one cache claims "exclusive access" to the data, no other cache will write to it (although they may read stale versions of that data from their own cache). When that core / L1 cache is done, it will "invalidate" the data.
Modern cache coherence protocols can go faster: MESIF for Intel means that writes may be (F)orwarded to other caches automatically (which is slightly faster than (E)xclusive -> (I)nvalid -> (E)xclusive in the 2nd core).
Again, this is all done automatically, because programmers want to have their data look the same from all cores (for some definition of "looks the same").
This does NOT mean that the data is in the correct order however (!!). L1 caches can change the order of reads and writes before the data is shipped to other cores. Indeed, this is the very POINT of L1 cache. If one core is doing a while(true) iter++; loop, its more efficient to merge the writes together and "batch" them out slowly. Ex: Core#2 may see "1, 10, 33, 600" each time it reads the value of "iter".
Indeed, the most efficient implementation of this while(true) iter++; loop is to have "iter" be a register private to a core that has absolutely no communication to the outside world at all.