The Spinnaker Project: Revolutionizing Brain Modeling with Machine Learning

My current research is focused on the Spinnaker project, a crude contraction of Spike Neural Network Architecture. The goal of this project is to build a machine that incorporates a million arm processor cores, which we can use for real-time brain modeling applications. With a million cores, we only get to about 1% of the scale of the human brain, as indicated by the example of a mouse brain, weighing around 10 whole mouse brains. This gives us an idea of the problem that we're taking on here.

The physical machine occupies or will occupy 10-19 inch racks, which are standard cases about 6-7 feet tall and bit over 90 inches wide. The machine will be built here in Manchester, with one of those racks already populated with a hundred thousand cores, equivalent to one mouse brain's worth in terms of network complexity. Modeling a real mouse brain requires lots of biological detail that we won't be going into and knowledge of the network that doesn't currently exist, but this gives us an idea of what we're working towards.

The design objective of Spinnaker was to model about a thousand neurons in real time on each arm core, so with a million core machine, we get to a billion neurons in total. At the moment, we're still tweaking the efficiency of software and the current software implementation maps about neurons to each core, so we aren't at the efficiency we're ultimately aiming for but we're close. The basis of human memory is not really well understood; computer memory is relatively simple, where we arrange a silicon structure and have a cell that can store a 0 or a 1 and know exactly where that is, and if the cell breaks, we lose that information. Human memory seems to be much more distributed, with all through adult life losing about one neuron per second.

This lack of understanding of how the brain works fundamentally as an information processor is what drives this work. We have each got this thing inside our heads that is clearly vital to the way we perform our daily tasks, but we don't know how it works or how information is processed at those levels. The only instruments we have for increasing our understanding are computer models, which have to be informed by what we do know about neuroscience and other disciplines like psychology and mathematics.

The Spinnaker architecture looks reasonably conventional; each node is an ARM processor with a certain amount of code memory and data memory. Programming the processor itself is also reasonably conventional because we're aiming for real time processing. We use what's called an event-driven programming model, where instead of having a single program that runs and loops around, the programmer writes a number of event handlers that get invoked when particular events happen.

For example, if an incoming packet arrives from another neuron, it calls a little bit of code that says, "What do I do when this input spikes?" The response to that would be to look up the synapse that is connected to that input and pass the relevant level of signal through to the neurons that are modeled here. This may or may not cause some of those neurons to spike and if they do, then it drops more packets into the fabric and so on. This event-driven programming model allows for efficient processing in real-time.

The Spinnaker project is a collaboration between disciplines like neuroscience, computer science, psychology, and mathematics. It's an interdisciplinary effort that aims to contribute to our understanding of how the brain works. By building a machine that can process information at a scale similar to the human brain, we hope to gain insights into how the brain processes information and how it stores memory.

The architecture of Spinnaker is also designed with efficiency in mind. Apart from the communications network, the Spinnaker architecture looks reasonably conventional; each node is an ARM processor with a certain amount of code memory and data memory. The machine was packed together sufficiently densely that it needs a bit of a draft to keep it cool.

The software implementation maps about neurons to each core, so we aren't at the efficiency we're ultimately aiming for but we're close. It took a long time and it turned out that spending four hours on an idea of what I could do that day is not as productive as starting with something simple and then gradually increasing complexity.

The Spinnaker project has taken a lot of time to develop, and it's still under development. The aim of building the Spinnaker machine was to contribute to the scientific ground challenge understanding how the brain works. We have each got this thing inside our heads that is clearly vital to the way we perform our daily tasks, but we don't know how it works or how information is processed at those levels.

"WEBVTTKind: captionsLanguage: enSo my current research is focused on the spinnaker projectSpinnaker is a crudeContraction of spiking Neural Network Architecture the goal is build a machineincorporating a million arm processor coresWhich we can use for real-time brain modeling applications now with a million cores we only get to about1% of the scale of the human BrainI to think of it 10Whole mouse brains which gives you some idea of the scale of the problem that that we're taking on hereThe physical machine occupies or will occupy 10 19-inch racks, so these areStandard cases about 6 7 feet tallBit over 90 inches wide and about a deep the machine will be built here in and be based in ManchesterWe this stage we have one of those racks populated so a hundred thousand coresWhich is about one mouse brains worth in terms of Network complexity course?modeling a real mouse brain requires lots of biological detail that's that we won't be going into andknowledge of the Network that doesn't currently exist, butJust gives you a feel of the scale of network that we can model on the machinethe physical design of Spinnaker is really reasonably conventional andWe have the chips each of which has 18 cores onWe mount 48 of those chips on the circuit board which is about the size of an A4 sheet of paperso that gives you the of a thousand processors on a circuit board and then we stack thecircuit boards in card frames, so they're plugged intointo standard racksQuite close together and which we can do because we've been very careful to keep the power consumption under controlSo there are no real on the board and therefore, it's quiteEasy to even when they're spaced quite closely together so can get 24 boards across the width of a 19 inch card frameAnd we can stack thoseframes high in A 19-inch rackIncluding allowing space for cooling and so on and and that gives you120 boards each with a and processors, which is a hundred thousand?over 100,000 processors in the 19-inch rackso with the PCBsLook pretty conventional. They're just lots of little black plastic packages and sold it together in a slightly unusual irregular arrayThe thing you'd notice about the board is it is highly regular I can show you one if you want it would love to seeOne yeah is the one yeah yeah can we show yeah, so this isOne of the Spinnaker boards this one's packagedin its own acrylic boxyou can see the spinnaker chips thethe name gives us anObvious logo of a spinnaker sail which is quite attractive and easy to put on the chip so you can see there are 48Chips on this board it may look they're arranged in a square, but actuallyTopologically they're in a hexagon, so the board is a hexagonal sub surface of the toroidthethe boards have high speed connectors on the Edges andThat allows us to connect multiple boards together to build the larger systemsSo when we put a hundred thousand cause in a rack all the boards in that rocker arranged on a single continuous toroidWhich keeps the communication efficient.So, so this is a inch card frame with spinnaker boards in twenty thousand arm processorsAnd you can see the boards are all linked together through high speed links which puts the chips into a continuous singletoroidal surfaceAnd then the boards are connected through a network switch to an ethernet system. Which isUsed for loading software onto the machine on the top of the boxthere's a robot with a single spinnaker card on its back the robot is built byyour conrad at the technical university of Munich and it has a couple ofvision Sensors from Toby dell Brook at the University of Zurich on the front and that can be used to build simple Neural robotcontrollers that run on the spinnaker boardThat's a . That's basically controlling the robot twice eight hundred and sixty four processors. Which is more thana robot will usually getAre we going now? I'll show you the big machine. That's sir. Yes, so we're walking through the it building here andSo this is a processor spinnaker machineit has five the card frames that weSaw in the previous shot now you can see the wiring that wires them together. That's got a lot more complicated and stringyThe primary problems is it all the wiring yes? It's quite complicated to wire it took a phd studenthours to wire the machine up in the front the machine is basically connected the ethernetSo you can control it remotelyYou need some kind of host machine to compile the network on and you download it over the net ontothe Spinnaker machine itself and thenThe thing then runs and the network evolves in real time even the small card frames a bit noisywere very careful about the power consumption thatIt was still packing it together sufficiently densely that it needs a bit of a draft to keep it coolApart from the communications network the spinnaker architecture looks reasonably conventional is it's a massively parallelComputer each node is an arm processor with a certain amount of code memory and a certain amount of Data memoryAnd so programming the processor caused themselves is reasonably conventional because we're aiming for real timeWe use what's called an event-driven programming model. So instead of having a single program runs and loops aroundThe programmer writes a number of event handlers which a little bit of little bits of code that get invokedWhen particular events happen such as an incoming packet arrives from another neuronThat in that calls a little bit of code which says wellWhat do I do when this input spikes and what I do is I?Look up the synapse is that are connected to that import?And I pass the relevant level of signal through to the neurons that modeled hereandthat may or may not cause some those neurons to spike and if they do then I drop more packets into the fabric andSo on the design objective of spinnaker was to model about a thousand neuronsIn real time on each arm core so with a million core machine we get to a billion neurons in total nowat the moment we're still tweaking the efficiency of software and the current software implementation maps aboutNeurons to each core so so we aren't at the efficiencyWe're ultimately aiming for but we're close the basis of human memory not really well, understoodI mean computer memory is relatively very simpleWe arrange a silicon structure and we have a cell where we can store a naught or a 1 and we know exactly where thatis and if the cell breaks we lose that informationHuman memory doesn't seem to be like that human memory seems to be much more distributedwe allthrough adult life losing about a neuron a secondAnd so if he any in your wrong was critical to a memory then each neuron we lost will be some memory goingAnd that doesn't seem to be the case it seems to be thathas found ways store information in distributed ways, so if you lose small componentsyou don't lose the memory the aim of building the spinnaker machine was to contribute to theScientific ground challenge understanding how the brain works we have we each have this thing inside our heads thatIs clearly vital to the way we perform our daily tasksBut we don't know how it works fundamentally as an information processor. We don't haveDeep Knowledge, we know lots about the fine detail about about the neurons and the cellular biologyAnd we know quite a lot about how activity moves around inside the head, but the interesting stuff aboutFor me as a computer scientist is how information is processedAnd that's the levels that we can't access either from the bottom up or from the top downSo the only instruments we have forincreasing our understanding arecomputer models now of courseThe computer models have to be informed by what we do know aboutNeuroscience, soso this work is very much a collaboration betweendisciplines and Neuroscience computer sciencepsychology, all come into play with quite a lot of mathematics -It's a little bit similar to the yellow oneso we put it'll be closer to the yellow one and then comes another data item and no this is obviously quite similar to theYellow one to be put it closer to heredon't want to just - something off, but IRead it off make that my sentences are well constructed and spelled correctly and so on so it took a long time and itTurned out that. I was spendingYou know I would start out with an idea of what I could do that day, and then four hours later\n"