The document describes the LinnStrument, an open-source musical instrument created by Geert Bevin. It uses Arduino boards and sensors to allow for touch-based music performance and expression in 3D. The document discusses the hardware components, including LEDs, sensors, and foot pedals. It also covers accessing these components through Arduino code and firmware, including touch tracking, MIDI output, and debugging tools. The goal is to make the instrument fully hackable and customizable through open-source software and tools.
17. ARM Cortex-M3
32-bit 84MHz
CPU
SPI signals
shared by
LED/Sensor
Digital 33-37
Footpedals
DIN <-> USB
LED control
MIDI <-> Serial
18. Arduino Due and LinnStrument
• 32-bit core, 4 bytes wide operations within single CPU clock
• CPU Clock at 84Mhz
• 96 KBytes of SRAM
• 512 KBytes of Flash memory for code
• Digital I/O pins
• Serial Peripheral Interface (SPI) pins with Slave Select
19. Very simple Arduino program
// the setup function runs once when you press reset or power the board
void setup() {
// initialize digital pin 13 as an output.
pinMode(13, OUTPUT);
}
// the loop function runs over and over again forever
void loop() {
digitalWrite(13, HIGH); // turn the LED on (HIGH is the voltage level)
delay(1000); // wait for a second
digitalWrite(13, LOW); // turn the LED off by making the voltage LOW
delay(1000); // wait for a second
}
20. Arduino code
• language based on C/C++
• concise reference with all language structures, values and functions
• Arduino IDE to get started
• the ‘setup’ function runs once, when the board starts up
• the ‘loop’ function runs at 100% CPU, over and over again
21. Only one execution thread
• what happens in the ‘loop’ function is all that’s happening
• if something takes too long, something else isn’t happening
• guerrilla coding tactics: powerful, short strikes and get out of there
• design your algorithms to be sub-dividable
• avoid dynamic memory allocation, it’s very slow
30. Easy LED functions
• Change the color and brightness of a single LED
void setLed(byte col, // Column of LED to be changed
byte row, // Row of LED to be changed
byte color, // Color of LED to be changed
byte brightness) // Brightness of LED (0, 1, 2 or 3)
• Light up a single LED with the default color and brightness
void lightLed(byte col, // Column of LED to be changed
byte row ) // Row of LED to be changed
• Clear a single LED
void clearLed(byte col, // Column of LED to be changed
byte row ) // Row of LED to be changed
31. Details of LED control
• LED control is done through SPI using pin 10, with mode 0, running at 21MHz
SPI.begin(10);
SPI.setDataMode(10, SPI_MODE0);
SPI.setClockDivider(10, 4); // 84MHz divided by 4
• Digital pin 37 is an output and connected to the LED driver chips
pinMode(37, OUTPUT);
• Write 32-bit data structure to SPI to control the LEDs, refreshed every 100μs
digitalWrite(37, HIGH); // enable the outputs of the LED driver chips
SPI.transfer(10, column, SPI_CONTINUE); // send column (left-shifted 2 bits + special bit 7)
SPI.transfer(10, blue, SPI_CONTINUE); // send blue byte
SPI.transfer(10, green, SPI_CONTINUE); // send green byte
SPI.transfer(10, red); // send red byte
digitalWrite(37, LOW); // disable the outputs of the LED driver chips
32. Easy sensor functions
• Send 16-bit word over SPI to touch sensor to set the analog switches
void selectSensorCell(byte col, // Column used by analog switches
byte row, // Row used
byte switch) // Switch to read X (0), Y (1) or Z (2)
• Read stable raw X value at the current col and row (returns 0-4095)
int readX()
• Read stable raw Y value at the current col and row (returns 0-127)
int readY()
• Read stable raw Z value at the current col and row (returns 0-127, 255 feather)
byte readZ()
33. Details of touch sensor control
• Touch sensor control is done through SPI using pin 4,
with mode 0, running at 21MHz
SPI.begin(4);
SPI.setDataMode(4, SPI_MODE0);
SPI.setClockDivider(4, 4); // 84MHz divided by 4
• Write 16-bit data to SPI to set analog switches (see ls_sensor.ino)
SPI.transfer(4, lsb, SPI_CONTINUE); // first byte of data structure
SPI.transfer(4, msb); // second byte of data structure
34. Read touch sensor data
• Touch sensor A/D input is using SPI through pin 52,
with mode 0, running at 21MHz
SPI.begin(52);
SPI.setDataMode(52, SPI_MODE0);
SPI.setClockDivider(52, 4); // 84MHz divided by 4
• Read sensor data
delayUsec(7); // wait for stable current after sensor
// control changes the analog switches
// delay different for each analog switch
byte msb = SPI.transfer(4, 0, SPI_CONTINUE); // first byte of sensor data
byte lsb = SPI.transfer(4, 0); // second byte of sensor data
int raw = (int(msb) << 8 | lsb) >> 2; // pack into int, shift from 16 to 14 bit
35. Reading the foot pedals
• Done in ls_switches.ino, modify this method to add custom behavior
void handleFootSwitchState(byte whichSwitch, boolean state)
• Digital pin 33 and 34, respectively for left and right foot switches, configured as pull-up
inputs (inverted inputs: high is off, low is on)
pinMode(33, INPUT_PULLUP);
pinMode(34, INPUT_PULLUP);
• Read the digital value of foot pedal states (typically every 20ms)
boolean leftPedalState = digitalRead(33);
boolean rightPedalState = digitalRead(34);
36. Details of MIDI / Serial - USB / DIN
• Setting digital switches changes the communication methods from the LinnStrument
to the outside world
• Digital pin 35 switches between Serial and MIDI
pinMode(35, OUTPUT);
digitalWrite(35, HIGH); // high switches to Serial input/output
digitalWrite(35, LOW); // low switches to MIDI input/output
• Digital pin 36 switches between USB and DIN connectors
pinMode(36, OUTPUT);
digitalWrite(36, HIGH); // high switches to USB input/output
digitalWrite(36, LOW); // low switches to DIN input/output
49. Core Files
• linnstrument-firmware.ino: global data structures, setup and main loop
• ls_displayModes.ino: illuminate LEDs for the different display modes
• ls_handleTouches.ino: driven by main loop, handles touch tracking
• ls_rtos.ino: primitive scheduler calling continuous tasks during delay
• ls_settings.ino: switch behavior from UI, store and recall settings
• ls_touchInfo.ino: encapsulate sensor data into touched cells
50. Low-level Files
• ls_calibration.ino: calibration procedure and data conversion
• ls_leds.ino: low-level communication with the LEDs
• ls_midi.ino: MIDI input, NRPN control, clock, output and queue
• ls_sensor.ino: low-level touch sensing with bias and curve
• ls_test.ino: debug functions and low-level reports
51. Auxilliary Features Files
• ls_arpeggiator.ino: arpeggiator logic, tied to internal MIDI clock
• ls_faders.ino: MIDI CC faders touch handling and data sending
• ls_font.ino: tiny, small, and big font display, including scrolling
• ls_lowRow.ino: continuous cell evaluation for low-row features, driven by
the main touch tracking
• ls_noteTouchMapping.ino: track MIDI notes to touched cells, mainly used
by arpeggiator
• ls_switches.ino: handles control switches and foot pedals
52. Support Header Files
• ls_bytebuffer.h: circular byte buffer with independent push and pop
locations, used by ls_midi.ino output queue
• ls_channelbucket.h: hands out MIDI channels from a bucket of
allowed channel numbers.
• ls_debug.h: debug macros and defines
• ls_midi.h: standard MIDI status codes, used by ls_midi.ino
54. Global variables
• byte sensorCol: the column number of the current sensor cell
• byte sensorRow: the row number of the current sensor cell
• byte sensorSplit: the split of the current sensor cell (0: left, 1: right)
• DisplayMode displayMode: the active display mode
(see DisplayMode enum in linnstrument-firmware.ino for the values)
55. Global functions
• TouchInfo &cell(): the last touch data for the current cell
• TouchInfo &cell(byte col, byte row): the last touch data for a cell
• FocusCell &focus(byte split, byte channel): the current cell that has
the focus for a particular split and channel (FocusCell is a structure
that just contains col and row)
57. Per-finger touch-tracking
• With a general purpose CPU, you’d model this as touch ‘sessions’
that are dynamically created and have a life of their own
• Too much memory churn for Arduino, too much book-keeping also
• Instead, have a touch state for each cell
• Transfer data between cells since we don’t support two fingers
touching the same cell
58. Inside ls_handleTouches.ino
void handleNewTouch(byte z) {
// ... snip ...
// check if the new touch could be an ongoing slide to the right
if (potentialSlideTransferCandidate(sensorCol-1)) {
// if the pressure gets higher than adjacent cell,
// the slide is transitioning over
if (isReadyForSlideTransfer(sensorCol-1)) {
transferFromSameRowCell(sensorCol-1);
handleXYZupdate(z);
}
// otherwise act as if this new touch never happened
else {
cellTouched(transferCell);
}
}
// similar for slide to the left
59. Check potential slide transfer
boolean potentialSlideTransferCandidate(int col) {
if (col < 1) return false;
if (sensorSplit != getSplitOf(col)) return false;
if (!isLowRow() && (!Split[sensorSplit].sendX ||
!isFocusedCell(col, sensorRow))) return false;
if (isLowRow() && !lowRowRequiresSlideTracking()) return false;
if (isStrummingSplit(sensorSplit)) return false;
// the sibling cell has an active touch
return cell(col, sensorRow).touched != untouchedCell &&
// either a release is pending to be performed, or
(cell(col, sensorRow).pendingReleaseCount ||
// both cells are touched simultaneously on the edges
abs(cell().calibratedX() - cell(col, sensorRow).calibratedX())
< TRANSFER_SLIDE_PROXIMITY);
}
60. Is the sibling cell ready for the transfer?
boolean isReadyForSlideTransfer(int col) {
// there's a pending release waiting
return cell(col, sensorRow).pendingReleaseCount ||
// the cell pressure is higher
cell().rawZ > cell(col, sensorRow).rawZ;
}
63. Perform the data transfer 3/3
// transfer the focus if this was the focused cell
byte channel = cell().channel;
if (channel != -1 && col == focus(sensorSplit, channel).col &&
sensorRow == focus(sensorSplit, channel).row) {
focus(sensorSplit, channel).col = sensorCol;
focus(sensorSplit, channel).row = sensorRow;
}
}
64. Finding the right MIDI channels
• Note-per-channel distributes up to 16 MIDI channels across notes
• Notes should reuse same channel as late as possible (release trails)
• Intuitively you’d scan all the active notes and determine which channel is
available for a new note, which is again too much overhead
• We use a bucket of available channels, channels bubble up or sink down
• A new note merely has to take the next channel from the top
• Fully encapsulated inside ls_channelbucket.h
65. Low-row functionalities
• Intuitively you’d detect a touch on low-row cells when it’s active
• Then evaluate state of every other cell and trigger behavior
• This is again too much overhead
• Instead keep track of low-row start/stop in a state machine
• Piggy-back when processing each cell in the main loop to evaluate
appropriate low-row behavior
67. Inside ls_lowRow.ino 1/2
void lowRowStart() {
switch (Split[sensorSplit].lowRowMode) {
case lowRowStrum:
lowRowState[sensorCol] = pressed;
break;
// ... snip, different for each low-row mode
}
}
void lowRowStop() {
switch (Split[sensorSplit].lowRowMode) {
case lowRowStrum:
lowRowState[sensorCol] = inactive;
break;
// ... snip, different for each low-row mode
}
}
68. Inside ls_lowRow.ino 2/2
void handleLowRowState(byte z) {
// this is a low-row cell
if (isLowRow()) {
// send out the continuous data for low-row cells
if (cell().velocity) {
// ... snip, different for each low-row mode
}
}
// this is a non low-row cell
else {
switch (Split[sensorSplit].lowRowMode) {
case lowRowStrum:
// uses lowRowState to correlate with column
handleLowRowStrum();
break;
// ... snip, other cases
}
}
}
69. Sending MIDI bytes
• MIDI was causing the LEDs to flicker
• Too much time was spent at once (need more guerrilla!)
• Created a MIDI queue to continuously send byte-by-byte from our
RTOS
• Arduino UART classes still caused problems: synchronous wait for
readiness when sending