The Information Technology Life Cycle

From the bad old days of Hulking Giant Mainframes to the era of Grid Computing and Cloud Computing, I've observed a distinct life cycle of Information Techology organisations. This little piece is intended to share my anecdotal observations. No properly controlled study has been conducted to validate these findings; they are merely intended as a framework that might guide future research.

It is critically important to note that no criticism is intended of the hard workers who labour in the IT coal mines. They are no more in control of the forces that drive the IT life cycle than their clients (or their managers). Rather, the life cycle appears to be an inexorable consequence of organisational ecology.

Phase 1. Someone goes out and buys a shiny new computer. The proud owner puts some software on it, and uses it to solve some problems. Life is good, and then...

Phase 2. The owner's colleague realizes that the computer provides an effective solution to a problem that the colleague shares. "Hey, can I borrow your computing resources for my problem too?" asks the colleague. "Sure," replies the owner, because most owners of shiny new computers are eager to help their colleagues (and show off that their shiny new computers weren't a total waste of money and time). Now two people are getting their problems solved. Life is even better. Perhaps even more people join in the fun, until...

Phase 3. The computer becomes too big to handle as a side project. Either enough people are using it that they interfere with the owner's own work, or they are starting to interfere with each other, or there has been a failure and people realize that hey, maybe backups would have been a good thing after all. However it happens, there are two outcomes: either the original owner winds up having his job redefined to comprise mostly the care and feeding of the computer, or else the original owner (or his boss) hires someone to attend to it. An Information Technology organisation is born. The newfound support from management allows even more colleagues to get their problems solved. Life is sublime. Except for the poor IT guy, who finds out that...

Phase 4. More and more users come to the IT organization with more and more problems to be solved. Not all of these users are good friends of the computer owner; now they are people that the boss sent over. The IT guy works really hard to solve their problems, but discovers that it's an uphill battle. Some users want one thing, and some want another. Everyone wants an unreasonable quality of service. Work still gets done, but life is deteriorating, until eventually...

Phase 5. The IT guy eventually, to preserve sanity, begins to realize that the real mission of Information Technology is to protect the computer from the users. Anything new runs the risk of breaking existing functionality, so nothing can be done for the first time, ever. New development becomes forbidden. Management starts buying products to lock down system configurations, lest some user change something. The IT department may be outsourced. Work still gets accomplished, because by now the system is capable of handling most of the business's processes. Nevertheless, businesses continue to grow and change, and the system becomes a progressively worse match to the real needs. Eventually, someone decides that in order to make progress, they have to...

Phase 1.Buy a shiny new computer (which by now has become so cheap that they might even do it on their own nickel). Put some applications on it to do the actual work. Use it happily. The cycle has begun anew.

The turning of the cycle leaves systems and IT organisations in its wake. I've been to sites where there is a Hulking Giant Mainframe running business applications like Payroll; a successor to a Department Minicomputer running things like Sourcing; a Scientific Computer Cluster serving a bunch of Engineering Workstations; a whole lot of Personal Computers running Office Automation applications; Web Servers; Database Servers; Application Servers; maintained by half a dozen different Information Technology organizations - all of them ossified, and none of them speaking to any of the others (or to the users) in anything but a sneering tone.

In the spirit of Departing From The Script, I wonder:

Is there a way to step off this endless cycle? Or is it as inevitable as the Software Upgrade Treadmill, and just another manifestation of the Wheel of Life?

The solution, offered by some organizations, of requiring all new computer purchases to go through the IT Department, attempts to break the cycle. This so-called solution merely breeds stagnation, discontent, and eventual mutiny. The mutiny, if successful, merely has the effect of returning the cycle to Phase 1 for the mutineers.

Read more...

Allen organ project - card reader enhancement is done!

The hardware for the card reader enhancement is built! Excuse me while I crow.



It sits there on the organ console, looking as if it was built to go there. Well, it was.

Mounting keyboard and display was a matter of screwing in a whole bunch of #2-64 screws and nuts:



Then wiring the circuitry was a pretty routine perfboard job. The front of the board (toward the front panel) has sockets for the keypad and display, and mounts the trimpot that controls display contrast. It also has most of the actual wiring. There's a little scorch mark on one wire that would not stay out of the way of the soldering iron. Nobody's going to see it inside the box, and it's not near anything that's going to short it, so I'm just going to leave it alone. Nobody's perfecʇ.


The back of the board (the side that faces the inside of the box) mounts the RBBB, the DIN jack for the PS/2 keyboard cable, and a 4-pin socket where the serial interface goes for flashing the firmware.


Of course, I wound up making up some circuit modifications while I was wiring it up.

First off, I rearranged the pin assignments to give a neater layout on the perfboard with the RBBB turned sideways.

Next, I added a 470 Ω resistor and green LED to provide a 2.5 Hz "blinky light" status indicator that shows the firmware's main scan loop is running. (I've half a mind to have it send the firmware revision number in Morse.)

Finally, I decided that driving the LED backlight for the display - which is rated for 240 mA at a 4V forward voltage - off of the voltage regulator on board the RBBB was a losing proposition. Instead, I put together a little current regulator circuit. The 2N3053 transistor (good for a couple of Watts of collector dissipation) is switched with base current supplied from an RBBB pin via the 2.2K resistor. The 2.7Ω 2W wirewound resistor (it's mismarked 2.4Ω, which is why it turned up in surplus!) senses the emitter current. When the drop across it exceeds the Vbe of the little 2N2222A transistor, the little transistor turns on and robs the base current from the big one. Result is that the LED is driven at a 240 mA constant current, and then is pulse-width modulated for brightness. It's both brighter than the 6.8Ω resistor made it, and more stable with supply voltage and temperature fluctuations. Only thing is that the 2N3053 pass transistor was getting a little warm (on the borderline of exceeding the safe operating area of the "fingertip thermometer"), so I added a little homemade heatsink cut out of aluminum foil from a pie plate in the recycle bin. This technique works pretty well for medium-size devices as long as you use lots of grease to get good thermal contact with the foil.


The board goes into the project box, held in place by the keypad and display sockets in two corners and #4-40 screws, nuts and washers on fiber standoffs in the other two corners.


So now I try to flash the firmware (with the revised pin assignments). And ... nothing! Nada! Bupkis! The TX light flashes a couple of times, and avrdude refuses to sync with the chip. Several hours of puzzlement ensue.

Oh, [Nixonian expletive deleted once again]. I already made this mistake once, with the other RBBB inside the organ! And I've gone and done exactly the same thing again. I have RX and TX interchanged. (RX on the Ladyada serial interface goes to TX on the RBBB, and vice versa. Only this time, when I correct it - still nothing. Yes indeed, I've blown the chip. It's pretty forgiving, but this was too much for it.

Oh well, I still have one more Atmega168, sitting in the Arduino. (And I've been wanting to upgrade that one to a 328 anyway). So I swap chips...

And the display shows the menus nicely, and programs the organ.



I won't say that the firmware is done, but it's getting there. The big pieces that I want to add are that there's nothing behind menu #5 (where I want to put Hammond drawbar registrations), and I want to add an interface for scaling stops (making them louder or softer). Aside from that, though, it's more usable than the card reader - I don't have to worry about cards falling apart and lamps burning out.

I'm a happy camper for the moment.

A big thanks to Steve for getting me started with his surplus Arduino stuff, and to Carl K2YR for the fabrication assistance (and circuit sanity checking and general willingness to serve as the teddybear that I explain things to).

And, of course, thanks to Mary Ann and Cathy for putting up with all this geekiness.

Read more...

Allen organ project - display terminal front panel

Once again, I've been spending time doing the project and not writing about it. I've come up with a front panel layout for the display terminal, and cut some holes in the project box.




I started by printing a template on paper and putting it on the panel with spray glue. That gave me all the cut locations neatly laid out. As I did the template, I even remembered to displace the display unit a little bit upward so that the pins of the display and keypad lined up to holes on 0.1 inch perfboard.

Next came drilling some 5/16" holes in the middle of the two rectangular cutouts that hold the keypad and display. Then it was crunch-crunch-crunch with an Adel nibbling tool making rectangular cutouts that were (intentionally) a shade too small. (I can cut away more material, but it'd be real hard to put any back!) Then, with a file and patience, I enlarged the holes until they gave a snug fit for the components. (The keypad has a radius on each corner, and I just made that with a Dremel tool).

I used the components themselves as templates to mark the locations for #42 holes for the 2-64 mounting screws that hold the keypad and display, and drilled holes to mount the perfboard and to provide screwdriver access to the display contrast control. The screw holes came out a little ragged, what with being drilled from the wrong side, but the screw heads will cover that up anyway.



The fit of the perfboard inside is "perfect enough." If you click through to the original photo, you can see the socket pins lined up with the holes. That'll give me a base to wire on. The two nylon studs are for mounting the RBBB. [UPDATE: Decided not to use them, and just mount the RBBB flush to the perfboard.]

I also machined a slot into the side of the box below the keyboard to hold the DIN connector for the PS/2-style keyboard cable, and a mating groove on the connector. That one worked out nicely. The connector feels like a rigid fit even without backing it up with hot glue, which I plan to do anyway.



And I got another RBBB built. I soldered a 6-pin female connector to go to the DIN connector breakout. This box draws its power from the DIN connector and has the data lines going to two of the digital inputs. You can see where I ground away the breakout board so that the slot in the project box can engage the groove on the connector body. (Grinding a FR-4 fiberglass board takes patience and a dust mask.)

Plugging the DIN connector into the LED controller, the pilot light on the power supply lights and the Vcc pins check out at 5V, so I guess I got that part right.


Read more...

Allen organ project - proof of concept achieved. Facepalm.

Now that the LED control is integrated with the card reader, it's time to see if it actually can program a stop!

Open up the organ, and in it goes. Take the breadboard display terminal, select "Blockflöte 8'" and send it over.

WOOHOO! First try!

Try again - Octa-Bell. "Bing bong bing." Kinura. "Blaaaaat." Festival Trumpet. "ta-ta-ta-DAAAA!"

Time to close up the organ and work on suitable packaging for the terminal.

Oh dear. Oh dear. Oh [Nixonian expletive deleted].

The two rotary switches above the card reader leave absolutely no clearance behind the lamp strip.
There's no room for the rearward extension of the board.

Oh dear.

Fortunately, there's plenty of room underneath the card reader. Back to the bench.

I cut the card reader perfboard in half, and rewire everything so that I have two halves joined with a strip of ribbon cable. I'm a little worried about switching noise getting into the audio, so I try to control it some by having each LED have an independent ground return on an adjacent conductor. (Desolder all the LEDs and resistors. Solder in both ends of all 20 conductors of the ribbon cable. Replace the LED that I overheated in the rework process.)

After all this rework, Frankenreader goes back in the organ and looks like this:


From the rear. The logic board extends a little way under the reader.


From the top. The ribbon cable clears all the hardware above.

Read more...

Allen organ project - integrating with the organ

With the LED control apparently working on breadboard, it's time to get it aboard the organ. Mounting the LEDs and resistors, the RBBB, the programming port, and the PS/2 connector is a fairly simple matter of perf-board construction. (You'll notice that I replaced the voltage regulator on board with a heat-sinked 7805. The current draw of the LEDs was making the itty-bitty 5V regulator that comes with the RBBB get, uhm, kind of toasty. The 7805 is nice and cool.)



The LEDs need to be on quarter-inch spacing, while the perfboard hole spacing is 0.1 inch. But quarter-inch spacing is easily done by mounting every other LED on a diagonal pair of rather than a h0rizontal pair. The row 8 LED (second from top in the picture) is mounted vertically, which displaces is 0.05 inch toward the rear of the organ.

With a little tweaking, the LEDs fit the holes in the card reader beautifully:

The LED pin numbering was a little inconvenient for the perfboard layout, so I did a quick firmware flash to renumber them.

And the PS/2 communications crapped out. I'm getting spurious interrupts.

After a couple of evenings of very frustrating debugging, I finally get it. When I put in the keyboard cable, rather than the short wires on the breadboard, I added enough capacitance that the LOW->HIGH transition got really sloppy (remember, these are open collector outputs!). Adding a line of code to drive the pin HIGH at low impedance for a brief period (just long enough to change the pin mode to INPUT) makes the transition clean enough that the spurious interrupts go away. The data transmission looks solid as a rock.

And as I was testing to make sure that the card slot was unobstructed, I realized.

The photocell for LED 8 is displaced to the rear. Not the front. So it's not in quadrature with the others. Instead, it gets fired independently when all the others are off. A quick bout of editing and another few firmware flashes. (Slow the clock down again, to watch the lights blink, and then speed it back up. Plus debugging.)

The firmware is now at http://kbk.is-a-geek.net:2303/ci/95f796cc4d.

And it's time to try this thing out for real!

Read more...

Allen organ project - lighting the LEDs

Last time, I breadboarded the LED controller and got its serial communications working. Now it's time to fire those LED's.

When the controller is idle, all the LED's are on, as if there is no card in the slot. (Among other things, this convention allows me to continue programming the organ with cards, if I wish.)



Wow, those LED's are bright.

Now, let's look in detail at what happens when a card is inserted and removed.

[Update: I was wrong about the quadrature signal. The next post explains.]

• First, the paper blocks all the lamps (LEDs are all off). This condition persists until column 6 lines up with the photoresistors.
• Next, for each group of three columns, the row 9 hole, and the data in rows 0-6, come into view. LED 9 and the data byte are lit.
• The row 8 hole comes into view early, because its photocell is displaced half a column. LED 8 lights.
• The data column leaves the photocells (LEDs 0-6 and 9 go off).
• The timing column leaves the photocell (LED 8 goes off).
• About 1 1/2 column of blank space (all LEDs off) passes over, and then the cycle repeats for the next fifteen data bytes.

Once all this is done, the LEDs stay off for a short pause (the card blocks the light from all of them, andt then the entire cycle repeats, reversed in time, as the card is withdrawn. Finally, all the lamps are lit as the card exits the reader.

http://kbk.is-a-geek.net:2303/ci/41508259dd has the code baseline that does this. With the clock rate slowed down to where my human eyeball can track what's going on, the data look right.

Now to move the thing onto the card reader and try it out on the organ!

Read more...

Allen organ project - communicating with the LED control

In the last post, I cane up with a schematic for the LED control for the organ card reader. It was obvious from the schematic that an Arduino with a prototyping shield would be total overkill (and an excessively large form factor), so I decided to use an RBBB instead. The RBBB is cheap, tiny, and fully firmware-compatible with the Arduino Diecimila or Duemilanove. Breadboarding the project is simple:

From left, the major components are a serial adapter for flashing the firmware, the RBBB itself, and the LED array. For the breadboard, I didn't bother to attach the PS/2 connectors, instead directly wiring pins 3/4 between the Arduino and the RBBB, and the power supply rails (note the jumpers at the right hand edge). The LED pins in the picture were chosen for convenience on the breadboard and aren't the final pin assignments on the reader.

The first task is to make the LED control respond to commands from the display unit. This means firmware changes for both the boxes, of course.

The code baseline for these changes is over at http://kbk.is-a-geek.net:2303/ci/ac332e3e60, if you want to follow along.

I'm letting the display box play "keyboard", and the LED control play "computer". I'm following the PS/2 keyboard protocol loosely (not bothering to implement things like retransmit request, and of course, keyboard scan codes will be meaningless in this application). But the wire protocol is pretty much the same.

When the line is idle, both ends have DATA and CLOCK pulled to +5 through (nominally) 15K resistors. (I'm using the internal pullups on the microcontroller.) We're simulating 'open collector' ports, so from the Arduino's perspective the HIGH state is:

   pinMode(pin, INPUT);
 digitalWrite(pin, HIGH);  /* high impedance with pullup */

while the LOW state is:

   pinMoode(pin, OUTPUT);
 digitalWrite(pin, LOW);   /* low impedance pull to ground */

When the keyboard sends a byte, the procedure is shown in the diagram below. (In this case, the terminal is sending an ASCII 'U', 0x55.)



The terminal begins by pulling the DATA line LOW, to request to send. It waits for at least 50 microseconds, and tests that the host has not pulled the CLOCK line LOW to forbid transmission. (The host can do this at any time, and the terminal checks before pulling it LOW and after returning it HIGH.)

The terminal then sends an 11-bit data frame. Each bit (except for the 'start' bit, which is already there) is placed on the DATA line on a LOW->HIGH transisiton of the CLOCK line. The host clocks the bits out on the falling edge of the CLOCK line, when the terminal is holding them stable.

In order, the bits are:

• 1 : Start - always '0'
  
• 2-9 : 8 data bits, present least significant bit first
• 10 : A parity bit, chosen so that there is an even number of '1' bits in bits 2-10.
• 11 : Stop - always '1'

Following the STOP bit, the DATA and CLOCK lines are left at 1, which is the IDLE state.

The clock speed can be anywhere from 10 to 16.7 KHz, so the transitions are spaced anywhere from 30 to 50 microseconds apart. The terminal just uses 'delayMicroseconds' to insert the necessary time delays. The LED controller attaches an interrupt to the falling edge of the clock, and has a timeout activated from its scan loop to detect incomplete data frames.

There are four possible errors that the controller can detect when a data byte arrives.

• Parity error - an odd number of '1' bits was present in the frame - the number should be even.
• Framing error - the stop bit was a '0' - it must always be a '1'.
• Timeout - the data frame did not complete within 150 ms.
• Overrun - there was no buffer space available to store a newly-arrived byte.

For ease in debugging, I decide to make the commands sent from terminal to controller be ASCII, newline-terminated. Initially, I envision four commands:

• 'S' - Set stop. The 'S' is followed by 32 hexadecimal digits giving the wavetable from the punched card.
• 'T' - Test. The 'T' is followed by arbitrary text. The command is simply echoed to the serial port. It's there for debugging.
• 'W' - Wake up. This command brings the controller out of its 'standby' state - which means turning on the LEDs and delaying briefly for the organ to detect that the card slot is unobstructed. No data are expected between the 'W' byte and the newline.
  
• 'Z' - Zzzleep. This command brings the controller into 'standby' after the display has been idle for 30 seconds. It turns off the LEDs to save power. No data are expected between the 'Z' byte and the newline.
  

A test harness using the 'Serial' class shows that all these bits are working. Onward to firing the LEDs!

Read more...

Allen organ project - the LED controller

Once again, I find myself with a list of things to put in the build log, to try to catch up to what I've been doing. I'm really, really lax about keeping this thing up to date, because - well, frankly, it's more fun to build than to write about building! But if I'm ever going to remember how my modifications to the card reader work, I'm going to have to write stuff down.

The last post described the operator interface box's electronics in nearly finished form. Now we need something to sit in the organ and fire the LEDs. It turns out that it looks pretty easy to do such a thing, because the existing card reader lamps are mounted on a little PC board that is just held in with a couple of 6-32 screws and spacers - and that board is just connected with a couple of wires that provide 8 volt power. Mounting to its mounting points looks a snap.


Taking off the board and looking underneath, I see that the "grain of wheat" lamps stick into holes in the reader case underneath. The holes look to be #29 drill on 1/8 inch centers , and there are holes for the unused card columns (where no lamps are present in the picture). #29 holes are a nice loose fit to T-1 (3.0 mm) LED's. The LED body will go in the hole easily, and the base will not. OK, so what I need is an array of 10 white-light LEDs in that size. As I noted previously, lamp number 8 is displaced toward the rear of the organ.

Aside: I tried a high-brightness green LED for the burnt-out one, but it didn't work. The fact that it didn't wasn't really a surprise. CdS photoresistors - which is what it appears that this device is using - mostly need blue light.

Not having anything suitable in junk, I went and ordered a bunch of white LEDs from All Electronics (Their part number is LED-83.) The data sheet for those says they're supposed to get 20 mA drive current. (Good! An Arduino output can drive them.) And they have a 3.5-4 volt forward voltage. OK, that says that we want to scale the resistor so that it'll source 20 mA when dropping a volt and a half - 68 or 75 ohms ought to do it. The resistor will just be dissipating about 30 mW, so an eighth-watter should be just fine. And I just happen to have ten 68Ω resistors in my junkbox.

The only other I/O that should be needed is some sort of communication with the display unit. My criteria for this: a reasonably thin cable, a fairly robust, small and readily available connector, and no tying up the on-board UART on either the terminal or the LED controller. (I want to have access to the 'Serial' library when debugging!) A little bit of headscratching, and I decide that an old PS/2-style keyboard connector should be perfect. The PS/2 keyboard protocol is fully synchronous (with the keyboard generating the clock), so there's no need for accurate timing on the receiver - just shift in the bits on an interrupt. (That also means that I want at least the clock line on an interrupt-capable pin, so pins 3 and 4 become the PS/2 interface).

With this much settled, I've got a complete schematic.


The pinout on the PS/2 port is standard with one exception: I've changed the Vcc lead from +5 to the unregulated 9V input. That'll give me a ready way to power the display terminal through the cable, even though there's an LED backlight (240 mA) to run.

I also do identical wiring for the PS/2 connector over on the display side. So now I've got a communication channel. Next comes programming that channel.

Read more...

Allen organ project - getting to "Hello, world"

Last time, we got the keypad running. This time, we want to make the display go. The display I chose uses the popular HD44180 controller chip, for which several other Arduino developers have written libraries. But, I observe:

1. This display is a little bit different from any of the documented ones. It turns out that the 4×20 display is actually a 2×40 display, with the third line being the continuation of the first and the fourth being the continuation of the second.
2. The four-bit LCD libraries that are out there aren't quite debugged - in fact, one that I examined writes the nybbles in the wrong order - how this ever displays anything but gibberish is beyond me.
3. I struggle a little with initialization: How to get the chip running in two-line 5×8 character mode, with a 4-bit interface?

Fortunately, solving these problems is a combination of programming and some moving of wires.

Getting the initialization right is the hard part, so let's start there. The issue here is that we're coming up from an unknown state. We don't know what's on the screen, where the cursor is pointing, or what the interface width is. So we need to command a bunch of things. And to do that, since we've wired only four of the display's data bus lines, we need to get the display into 4-bit mode. OK, that's not too hard: writing the nybble 0x2 should do it - change the display from 8-bit to 4-bit, and we're good to send 4-bit commands.

Uhm, no. What if the display was in four-bit mode already? Then we need to write two nybbles to the command register. OK, 0x2 0x8 looks right (4-bit mode, 5x8 characters, two lines).

No, that's not right either! If we started in 8-bit mode, writing the 0x2 will set 4-bit mode, and then 0x8 will be half of the next commmand.

OK, now a light begins to dawn. All of the libraries, including LCD4BIT, begin by setting 8-bit mode several times, delaying in between sending the commands. And their comments don't make it clear why: they all seem to say things like, "the data sheet doesn't say anything about this, but other people do it, and it works." But I get it now: it's to bring the display back to a known state! We could start in one of three possible states: 4-bit mode, 4-bit mode with one nybble of a command sent, or 8-bit mode. Let's just keep sending 0x3 nybbles, and see where it gets us.

Starting from 8-bit mode: 0x3 is interpreted as 0x30 (the other nybble is grounded), and the display goes to 4-bit, 5×8 characters, 1 display line. Then we do the same thing, twice more, leaving the display in that state.

Starting from 4-bit mode: 0x3 0x3 is 0x33, which is still 4-bit, 5×8 characters, 1 display line. (The two least significant bits are ignored.) Then the third 0x3 command is interpreted as 0x30 in 8-bit mode, which is the same as above.

Starting from 4-bit mode with an incomplete command: 0x3 provides the last byte of whatever was being done - taking some unknown action, but that doesn't matter, since we're going to be reinitializing everything. At this point we need a delay to wait for the slowest possible command to finish. The next 0x3 0x3 is a new command, setting 4- bit, 5×8 characters, 1 display line.

So at the end of this sequence, wherever we started, we have the display in 8-bit mode. Now we can put it into 4-bit mode and have everything happy. We just send an 0x2 (interpreted as an 0x20 byte), and now it's 4-bit, 5×8 characters, 1 line. Now from 4-bit mode, we send 0x2 0x8 (interpreted as 0x28) and the display goes to 4-bit, 5×8 characters, 2 lines. Except that when I test it out, it doesn't. Everything else in initialization works, but the display stubbornly remains a 1-line display. Finally, I reread the description of this command in the datasheet, and see the note:

Perform the function at the head of the program before executing any instructions (except for the read busy flag and address instruction). From this point, the function set instruction cannot be executed unless the interface data length is changed.

So how do I set character size and display lines for a 4-bit interface? At this point, I'm tired, and disgusted by the revolting mess, and just crawl into bed.

Fast forward to the shower the next morning...

It occurs to me that DB0-DB3 are ignored in 4-bit mode. I don't need to ground them. And since the only 8-bit mode command I execute intentionally is to change the display mode (either to 8-bit or to 4-bit), I can just hard-wire in the nybble I need. Knowing that there's something I can try raises my mood. I finish the shower singing Sailing 'round Yarmouth, and head off to work with a smile.

Sure enough, when I get home and try wiring DB3 to +5 instead of ground, what do I see with the code in the ZIP archive of auxiliary files?

My project said "hello" to me! I hack in a digital brightness control (even remembering to make its response nonlinear to match the eyeball more closely), and now it can read a keypad and respond to it without being tethered to the computer. Enough tinkering for one evening, now I can spend a little time with the wife and daughter. Yay.

As an afterthought, I also program a few characters into the CGRAM. Describing the length of organ pipes requires at least the vulgar fractions 2/3, 3/5, 1/3 and 1/7. So I know I'm going to want appropriate characters for that. That'll save a few characters of precious screen real estate for each pipe description.

And as a further afterthought, I notice that every time I reset the Arduino, several characters of rubbish show up on the display. A bit of thought reveals that the cause is that the bootloader is flashing the LED on pin 13, which is the same pin that I chose for the ENABLE line of the display. ENABLE gets moved to 14. Another minor annoyance is licked.

Read more...

Allen organ project - reading the keypad.

The first thing that I tried with the organ project was some code to read the keypad. I looked at http://arduino.cc/, and while I saw a few suggestions, nothing did quite what I wanted. No matter, reading a keypad isn't that hard.

It turns out that the Atmel chip that the Arduino uses is great for this function, because it's got the capability of putting a pull-up resistor on its inputs. So there's no need for any external components - all the rows and columns of the keypad can just be connected with wires. Here's the plan for discovering what keys are pressed at any given moment:

• Initially, configure all the rows and columns as inputs, but put pull-up resistors on the rows by writing HIGH to them as inputs. In this quiescent state, reading any row pin will see a logic HIGH.
• When polling the keyboard, switch column 0 to be an output. This will put it into a low-impedance state, and overwhelm the pull-up resistor for any pin where there's a key pressed in column 0. Read each row pin in turn, and record what key, if any is pressed.
• Set column 0 back to be an input and column 1 to be an output. Then do the same thing with column 2.
• If no key is pressed, return a constant KP_NOKEY. If multiple keys are pressed, return a constant KP_ROLLOVER. Otherwise, return the number of the key that was pressed (0-9, or one of the constants KP_ASTERISK or KP_OCTOTHORP).

The relevant source code is the kpInit() and kpState() procedures in the file 'terminal/terminal.pde' from ZIP archive of auxiliary files.

Now, it turns out that to read a keypad you need to do a little bit more than just poll the state. You also have to deal with key bounce, which is the fact that the key switch will often open and close many times when being pressed or released. You also have to deal with key rollover, where the operator presses a second key before releasing the first one. (The code I'm presenting here implements "2-key rollover", which handles only the case of two keys pressed at once. Full "n-key rollover" requires diodes in series with the key switches, which this cheap keypad doesn't have. Fortunately, n-key rollover is mostly needed for typewriter-style keyboards and touch typists.)

The kpPoll() function in the source file handles both of these. It is intended to be called often from the scan loop. It determines the current time, and reads the keyboard state. It ignores what it has read, unless the state has not changed at any time during the last 20 milliseconds (KP_DEBOUNCE_MILLIS). This check provides the "debouncing" function that makes sure that key bounce doesn't cause redundant inputs. If the stable state is now different from the last stable state, it's returned - unless it's KP_ROLLOVER, which indicates that multiple keys are pressed at once, or KP_NONE, which indicates that no key is pressed at all. So the return of 0-9 or of KP_ASTERISK or KP_OCTOTHORP indicates that a key was just pressed, and demands response to a user action.

And that's pretty much all there is to reading the keypad. Since the pins are in a scrambled sequence, I keep a little table in PROGMEM of where the rows and columns are, and what order the keys are in. There are also three RAM variables: the last state of the keyboard, the last state that was stable for 20 ms, and the time at which the state last was observed to change.

Next up: writing the display. That bit was a little trickier.

Read more...

Allen organ project - breadboarding the display terminal

As a first task toward making the card reader enhancement, I decided to start breadboarding the display terminal that will go on the outside of the organ to give me access to the alterable stops.

Breadboard with (from top) RBBB serial programming interface; RBBB for driving LED's; 12-key keypad; Arduino with LadyAda prototyping shield; LCD contrast pot and backlight components; 4x20 LCD.

Earlier posts in this series:

• Allen organ project
  
• Allen organ project - introduction
• Allen organ project - reverse engineering the card format
• Allen organ project - a software card reader
• Allen organ project - catching up

As a temporary mount, I soldered male headers on the keypad and display, and plugged them into a solderless breadboard. I then started wiring up the pins. First comes the keypad.

The two end pins on the keypad have no internal connections, and are not counted in the manufacturer's pin numbering. The remainin pins are:

Keypad Pin	Function	Arduino pin
1	Column 2	6
2	Row 0	5
3	Column 1	4
4	Row 3	19
5	Column 0	18
6	Row 2	17
7	Row 1	16

You can see from the table that I wired these "around the horn",at one end of the Arduino, skipping over pins 0-1 (the serial port) and 2-3 (which support interrupts, not needed to run the keyboard but maybe needed for something else).

I also plugged in the display at this time. Walking down through the list of pins on the display and trying to decipher the East Asian-flavored English in the datasheets, I wired it up:

Display Pin	Function	Arduino pin
1	Vss	GND
2	Vdd	+5V
3	Contrast adjust	See below
4	Register Select	13
5	Read/write	GND
6	Enable	12
7	DB0	GND
7	DB0	GND
8	DB1	GND
9	DB2	GND
10	DB3	See below
11	DB4	10
12	DB5	9
13	DB6	8
14	DB7	7
15	LED+	+5V
16	LED-	See below

The contrast adjustment went to the wiper of a 10k pot connected between +5V and ground. Initially, I grounded DB3, but we'll see in a subsequent post why that was wrong. Finally, there was a need for backlight control. You'll noticed I skipped over pin 11 in the table above - that's because I needed a pulse-width modulated output for the backlight. I connected pin 11 through a 1k resistor to the base of a 2N3904 transistor. The emitter went to ground, and the collector went to the LED cathode through a parallel pair of 12Ω resistors. (The data sheet showed 6.8Ω in the test circuit, but I didn't happen to have that size in the junk box.)

With suitable programming, this was enough to light up the display, read the keypad and put gibberish on the screen. The next post or two will get us from gibberish to "Hello, world!" and show how the keyboard is read and the display is managed.

To handle them, you'll probably want to download the ZIP archive of auxiliary files from this project's Fossil repository. You'll find the schematic in there under the name terminal/display.sch (Eagle format) and terminal/display.pdf (Adobe PDF).

Next post: reading from the keyboard.

Read more...

Allen organ project - getting stuff.

I also made a little time to do some rough-cut design for the card reader enhancement, at least enough to order some parts. The idea will be to have an Arduino in a box outside with a keypad and display to do operator interface (or perhaps "organist interface"), and a little box piggyback on the card reader to drive the LEDs.

Earlier posts in this series:

• Allen organ project - introduction


• Allen organ project - reverse engineering the card format


• Allen organ project - a software card reader


• Allen organ project - catching up

I was going to just put the LEDs in the inboard box, and drive them through a printer cable or some such, but my friend Steve made the good suggestion to run them off the RBBB that he gave me, and have a serial link joining the two boards. There's lots less wire that way.

I have on hand an Arduino (with the LadyAda protoshield), an RBBB (with the PC serial interface), and a suitable 9v wall wart - all also thanks to Steve, who had them as scrap from another project that turned out to be a little too big for them. So I'm tentatively looking at a block diagram like this:

The operator interface will use a telephone keypad for the input. It needs a reasonably large display, because there's a bewildering array of stops, with names like "Trompette en Chamade 8'" that need to be presented. So I'm going with 4 lines of 20 characters, which is the largest that the inexpensive HD44180 controller will support.

I need some reasonably bright white LEDs, 3mm diameter. (5 mm is a trifle too large to go into the holes on the card reader.)

So, significant items on the shopping list are:

• The LED's - 10 needed, plus a few spares.


• The display - it looks as if SparkFun's LCD-00256 (actually a Xiamen Ocular GDM2004D) is Just The Thing.


• A telephone keypad - I've got one in junk, but it's really crappy looking and I'd like the finished project to look halfway presentable.


• Some suitable connectors for the serial link. I'm also planning to power the operator interface box over those connectors. I'm thinking that a 6-pin mini-DIN, like a computer keyboard, should work nicely. And I've got a spare cable for that form factor.


• Oh yes, and some bits and pieces that I'm running short on (pin headers and the like).

(And to be honest, I ordered all of this stuff last week, and just didn't trouble to blog about it.)

Read more...

Allen organ project - catching up (UPDATED: Now with photo goodness)

Contrary to what you might think, I haven't been exactly idle on the Allen organ project. A few words on what I've been up to are probably in order.

Earlier posts in this series:

• Allen organ project - introduction


• Allen organ project - reverse engineering the card format


• Allen organ project - a software card reader

First of all, I've been busy getting the house ready to accommodate things without having a rat's nest of cables on the music room floor. This involved fishing two strands of speaker wire and a strand of Romex to each speaker. The two strands of speaker wire were for the audio and for 12VDC. The strand of Romex was for switched house current. That's what runs the motors that give the Leslie-like "gyrophonic" effects.

What's behind a speaker - Mini-Twist ML3 receptacle, an existing outlet, and a Keystone panel for audio and DC.

And behind the organ console, I fished up the other ends of all this. So behind each speaker I've got a Mini-Twist ML3 receptacle (I didn't want anyone plugging a lamp into it) and a 4-binding-post Keystone panel. Behind the organ console is a 6-binding post Keystone panel (left speaker, right speaker, and 12VDC to both), and a Mini-Twist ML2 inlet. (An inlet puts power into the wall, just as an outlet takes it out.) The arrangement that they had at the house where we got the organ was scary - the inlet was a lamp plug put on an outlet plate inside-out with some sort of putty; the low-voltage wiring was in the same box with the AC, and so on. The arrangement I have now at least isn't going to haunt my dreams.

What used to be behind the organ console - a cheap lamp plug mounted in reverse using some sort of putty, and two weird 6-pin speaker plugs, also puttied in place. How many code violations can you find in this picture?

The outlets and inlet were designed for panel mount rather than outlet box mount, so I made up nickel-plate switchplates with cutouts for them. A big "thank you" to Carl, K2YR for having the right Greenlee punch for the job!

As I did the ML2's for the various line cords, I discovered that all the AC wiring was badly deteriorated - the gutta-percha insulation unside the line cords crumbled at a touch. So I replaced all that, hacking up a hardware-store extension cord because that's a lot cheaper than the same length of 14 gauge stranded cable. I surely don't know why.

And while I was working inside the speakers, I noticed that the belts that drive the moving parts were also rotten - so I'm going to have to order a new set. OK, so I disassembled the shafts, so that I could unthread the belts to measure them. And as I was taking the brushes off the slip rings, I discovered that they were broken. Oh joy, that's a job that I'll have to get a factory service tech to do - because I can't find that part anywhere. It's a silver graphite brush, 0.125" by 0.25" by 1" with a bevel for the slip rings and a long spring. Oh well, brushes ought to be a maintenance part, so I bet the Allen tech can still get them. I hate to call a tech in for a job I can do myself, though.

So I soldered in a bypass to get power to the speakers (with quick disconnects everywhere so that I just need to yank it out to have the old arrangement back). I can live without the motion effects for a while.

I also tried replacing the burnt-out lamp in the card reader with a junk-box green LED (and current limiting resistor). Nothing doing; the thing still doesn't read cards. Next step on that device will be to replace the green LED with one of the white ones that I just got. More on that in the next post.

Pictures another time. My daughter has got my camera. [UPDATE: I got it back.]

Read more...

Allen organ project - a software card reader [UPDATED]

Earlier posts in this series:

• Allen organ project - introduction


• Allen organ project - reverse engineering the card format

First things first. I can live without alterable voices for a short while. But these IBM cards are getting yellowed and brittle. They're 35 years old, and it's not going to be many more years until they crumble to dust. How to get the data off them? Transcribing by hand, the way that I did the first couple is not going to work. I know myself, I'll make too many errors and get eyestrain. There has to be a better way.

I don't want to make any electical modifications to the organ until I know more about it. I don't want to find myself frying some key irreplaceable part. So cobbling some interface to the card reader on the organ is Right Out. Similarly, the handful of services that are still out there who will read card decks are getting Really Expen$ive - I don't want to go that way either. So, what do I have that will read cards?

How about my document scanner? It'll read anything. (No, I haven't scanned my nether regions. It's been a long time since I was a college kid, and about as long since I'd have thought that was funny.) But I have experimented with scanning photographic negatives (only mixed success), coins (too much reflection), and pressed flowers (left residue that was hard to scrub off the glass). Maybe I'll have better luck with IBM cards.

Immediately, I make the executive decision to put the card on the glass face up. All the printing on the card will add visual clutter and confuse any image recognition that I try to do. The back of the card is cleaner, albeit not nearly pristine. But let's give it a try. I also decide to try to give the card a clean, contrasting background. I riffle through my stash of coloured papers and pull out a sheet in robin's-egg blue that contrasts nicely with the yellowed paper of the card. I set the scanner for color at 150 dpi resolution, push the button, and out comes:





Not bad, there's plenty of contrast. There are also a lot of specks of schmutz on the card. Let's hope that doesn't make for a bad read.

Believe it or not, from here things ran pretty much in a straight line. Everything I tried worked once it was debugged. (Of course, I have had to do computer vision at several points in my career, so my educated guesses about what to try are usually pretty good.) Here's an outline of how the program took shape:

1. Segment the image.
I used the k-means algorithm to divide the pixels into three categories: cardboard, background and schmutz. To seed the clusters, I started with an initial guess that cardstock is yellow, background is cyan, and schmutz is black. The algorithm converged quickly (only about ten iterations), and resulted in a segmentation that looked quite usable:



(In the image, white is cardboard, black is background, grey is everything else.) Again, it looks usable; the vast majority of pixels are classified correctly.

2. Extract edges from the image.
Given the segmentation, extracting the edges simply means highlighting pixels that differ from their neighbours above or to the left. Easy.

3. Find straight lines among the edges.
We're trying to find a frame of reference for the card so that we can lay the grid of holes over it, despite the fact that the position on the scanner glass was uncertain. A first step is to find straight lines. To do this, I use an integral transform called the Hough transform.

The Hough transform works by shooting bundles of parallel rays through the image, at angles ranging from zero to π. It counts the number of pixels marked as being 'edges' on each ray.

Update: One colleague called me on the lack of mathematical rigour in this discussion. Another praised me for an immediately-graspable physical analogy. To the first, I say: I could have begun the discussion with a sentence like, 'The Hough transform is the integral,

.'
But I didn't."

The result is a plot of pixel counts, with the horizontal axis being the angle of the ray and the vertical axis being the distance of the ray from the centre of the image. The rays that go right down the edges of the card have a great many edge pixels on them, and show up as brilliant white spots in the plot in Hough space.



In this plot of a card in Hough space, you can see the angle of the short edge of the card as a vertical column of points near the center of the image. The short edges of the card are represented by brilliant white spots near the top and bottom. You can also see the spots representing other vertical lines on the card (vertical rows of holes) in the same column, giving us a way to measure hole spacing, since the vertical axis is measured in pixels. The long edges of the card fall in a column near the right-hand border, and once again you can see other lines (the card rows) of 'edge' pixels in the same column (and therefore representing parallel rays). These are the rows of the card, and again give us a convenient ruler for laying out our grid.

4. Lay out a grid parallel to the card axes.
Given the maxima in the Hough image, which represent the card edges, it's a simple matter of trigonometry to locate the card corners and create a grid to search for holes, given the known dimensions of an IBM card.

Update:A colleague asked me how robust this process is in the face of card misalignment. I took a near-worst-case misalignment (the card canted about 30° from the correct angle), and did a quick hack to the code to overlay the computed grid in red. It's not perfect, but that's mostly because the card edges are modelled as parallel lines - and are not quite parallel (or indeed, quite straight) in real life.

5. Count up how many 'background' pixels are in each hole location, and threshold on the background-to-total-pixel ratio.
This operation gives us the location of the holes. And it worked first try! (Well, once I fixed the typos in the dimensions of the card, it worked. The algorithm was fine, the numbers were wrong. Garbage in, garbage out.) As a little bit extra, I added a decoder for the Hollerith data at the right side of the card (Allen's part number, plus some number whose significance so far eludes me.) The resulting program prints out an ASCII picture of the card, and saves the 16 words of binary data.

         11111111112222222222333333333344444444445555555555666666666677777777778
12345678901234567890123456789012345678901234567890123456789012345678901234567890
--------------------------------------------------------------------------------
      8  8  8  8  8  8  8  8  8  8  8  8  8  8  8  8               20.0 S00R1305
                                                                     ]          
                                                                           ]    
              ]        ]        ]  ]     ]        ]        ]        ] ] ]]]   ] 
     ]     ]  ]  ]     ]           ]           ]  ]        ]                ]   
     ]  ]  ]        ]  ]        ]        ]     ]           ]       ]    ]       
           ]     ]     ]  ]     ]     ]        ]  ]        ]         ]       ]  
           ]           ]     ]  ]  ]        ]     ]        ]                    
     ]           ]     ]              ]  ]        ]        ]                   ]
     ]           ]     ]        ]     ]           ]        ]                    
                                                                                
      ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]                 ]          
     ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]  ]                        ]    
--------------------------------------------------------------------------------

Beautiful, it exactly matches the punches. Now to try recovering more cards!

Read more...

Allen organ project - reverse engineering the card format

Earlier post in this series:
Allen organ project - introduction

It looks as if a necessary first step here is to figure out the format of the data on the cards. Fortunately, the data didn't seem too difficult to reverse engineer.

Here's a bad picture (I'm far too lazy to shoot a better one, since this shows what we need to see) of two of the voice cards. To my ear, both of them sound nearly like sine waves, the Blockflöte at eight-foot pitch and the Flûte at 4-foot pitch.

The first thing I notice is that the large blank area at the middle right of each card is where the reading stops. (The card doesn't go into the reader any farther than that.) So only columns 6-52 (1-5 are blank) appear to be significant for the voice. (The remaining columns are Hollerith-encoded and appear to have a part number for the card and perhaps a few characters of additional information).

The bottom two rows, rows 8-9, of each card appear to have some sort of timing information. (See below - there's more insight to be had here.) On every card, they appear in a regular pattern: 9, 8, space, 9, 8, space.

The only other rows that are punched are 0-6. 7, 11, and 12 are always blank. On the Blockflöte card, you can sort of see a sine wave pattern running in the shape of the punches. This immediately made me guess that row 6 is the most significant bit of a 7-bit binary word. Plotting out the Blockflöte with this assumption gave a graph like:



Looks as if I guessed nearly right, except that this is only half a cycle. Let's see what happens with the flute. Here, row 6 is doing something strange, but I notice that the blank columns above it are moving in a sinusoid? Making the guess that it's two's complement, I plot out the binary numbers there:



Wow, a perfect sine wave!

I'm going to guess from the half-cycle of the 8-foot stop that the electronics in the organ use symmetry to make another half cycle from the samples that are supplied. For the resulting curve to be continuous and smooth, the second half cycle would have to be both inverted in level and reversed in time, so that the complete Blockflöte waveform would be something like:




We'll see whether I'm right if I ever try to make my own voices.

Now what about those timing tracks? I'd have a hard time designing circuitry to clock accurately a card being pushed and pulled by hand - it would be like clocking hand-sent Morse code (a difficult problem!). But wait a minute. Suddenly I remember that when I took a look at the card reader, the lamps seemed slightly
askew. It seemed awfully sloppy construction for a top-of-the-line (for its time) organ like this one. Looking back at the picture, I see:





I can see that rows 11 and 7 are unused, as I expected. Row 12 has a lamp over it; I'm guessing this row is used as an indicator that there's a card in the slot.
Certainly, with that lamp burnt out, no cards are reading, even if they have
no punches in row 12. Row 8 is displaced by about half a column from the others? What will this do to the relative timings? Aha! With the displacement, the row 8 signal will slightly overlap the row 9 one, giving a quadrature encoding that should be completely self clocking! Clever fellows, those Allen engineers. I'll have to remember that when I start blinking LEDs at the thing. I also make an educated guess that the reader is designed to operate on the pull, rather than the push. It's nearly impossible to push the card in evenly, but it's easy to pull smoothly.

Read more...

Allen organ project - introduction.

Those who know me will most likely already have heard that this spring, I bought an Allen church organ - one of the very earliest digital organs made. (A similar one is in the Smithsonian; the model is that significant to the history of organs.)

Well, let's just say that it's a very fine instrument, but showing its age in several small ways. Like any engineer, I have the attitude: "If you don't like the world, modify it!" So I've got a big fun project to keep me occupied for the summer. (In whatever spare time I'm not spending in trying to learn to play the thing!). So let's have a look at what I'm looking to do.

One particular period feature of the organ is that the alterable voices (wavetables in the early 1970's were novel technology!) are programmed with IBM cards, inserted manually into a little slot to the right of the manuals. Each card has a binary encoding of the waveform produced by a specific style of organ pipe.



I have dozens if not hundreds of cards for it, and they're getting rather fragile. 35 years ago, they weren't making punch cards to last for the ages. Moreover, the card reader has acted up twice for me. The first time, it was just dust and dirt obstructing part of it, and a can of air repaired it nicely. More recently, it's just quit. Time to investigate.
Getting to the card reader was a fairly simple matter once I found the two screws (one on each side) that hold the lid of the keydesk in place. The lid then swings up. If you try this, make sure you latch the music desk in the 'down' position first!

Once the lid is up, the stopboard is held in with two screws, one under the Pedal stoptabs and one under the Great.




With these screws out, it swings up out of the way.





Now the Swell manual simply lifts up:



Lifting the Swell manual reveals the card reader underneath it. The reader is held in with a card-edge connector and four screws. A quick test with the power switch reveals that one of the row of incandescent lamps on the top of the reader is burnt out.

Well, obviously that is one piece of technology that can be upgraded. Put LED's in there instead, and never worry about another burnout. But now a light bulb (or perhaps an LED, this is the twenty-first century) goes on over Kevin's head. This little lamp bar, easily removed, is the key to eliminating the deteriorating IBM cards without any electrical modifications to the organ. Pull out the light bar, and replace it with an LED bar with the LED's sequenced by a microcontroller as if the card was being inserted and withdrawn. And thus is born the Allen Organ Card Reader Upgrade.

Few good ideas are truly new. A little bit of trolling through the search engines shows that this idea is the subject of U.S. Patent 4,416,177. Fortunately, that patent is expired, and so there should be freedom to operate here. I'm essentially replicating exactly the device shown there, only with modern components.

Read more...