tag:blogger.com,1999:blog-135465162024-03-06T23:53:15.521-06:00Dan McCreary's BlogRuminations on NoSQL, XRX, XQuery, Semantics, STEM, Arduino, Internet-of-Things (IoT) and empowering the non-programmer.Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.comBlogger76125tag:blogger.com,1999:blog-13546516.post-5447158520122679262017-03-04T20:25:00.001-06:002017-03-05T08:58:38.842-06:00A Moving Rainbow Arduino Sign<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjEM5cWpoXB8C9bDIW7svIR1mn9cvwlkGOsLgNVOI9PP6hTp1VPbPiGiGCc70MKB9Rj_R9DcbsJUJcqEiNby48JXo3wwEOpEtv2xqm6RRYKF0HlgLW2EUgxIwNOAYqiEoheDQq7Q/s1600/arduino-led-sign.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="85" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjEM5cWpoXB8C9bDIW7svIR1mn9cvwlkGOsLgNVOI9PP6hTp1VPbPiGiGCc70MKB9Rj_R9DcbsJUJcqEiNby48JXo3wwEOpEtv2xqm6RRYKF0HlgLW2EUgxIwNOAYqiEoheDQq7Q/s320/arduino-led-sign.png" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">When I volunteer as a programming mentor at <a href="http://coderdojotc.org/" target="_blank">CoderDojoTC</a> we are in a large room with about 70 people. Some of the tables are setup for students learning to program Arduinos. When students first arrive in the room it is nice to have a sign directing them to the right tables. Since I know that color and motion are great ways to get kid's attention I decided to create a colorful programmable sign using addressable LED strings.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Since I made the sign I have had multiple requests from teachers and other Arduino mentors for instructions on how to make their own sign. These signs are ideal for display cases of a the science area of a school or a STEM program. So here are the detailed instructions on making your own sign.</span><br />
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Design goals</span></h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I had three design goals for this sign:</span><br />
<ol>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Use standard components that are easy for teachers and students to program with any moving color patterns that they want</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Make it low cost (under $20)</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Make it simple to construct - it should take about an hour once you have all the parts</span></li>
</ol>
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Parts</span></h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Here are the parts I used:</span><br />
<br />
<ol>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">50 pixel LED string ($12) <a href="http://www.ebay.com/itm/50pcs-WS2811-RGB-Digital-Color-12mm-Pixels-LED-String-Addressabble-IP68-5V-BK/371882178186" target="_blank">sample on ebay</a></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Scrap Plexiglas - 20"X 5" and 1/4 inch thick</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Arduino Nano ($3) <a href="http://www.ebay.com/itm/Mini-USB-Nano-V3-0-5V-16M-ATmega328P-CH340G-Micro-controller-board-For-Arduino/232045286935" target="_blank">sample on ebay</a></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">USB cable ($2) <a href="http://www.ebay.com/itm/Black-2-0-USB-Cable-Type-A-to-Mini-B-Male-To-Male-5-PIN-Charger-Cable-For-Phone-/272361596849" target="_blank">sample on ebay</a></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">1/2 size 400 connector solderless breadboard ($1.50)</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Two momentary push buttons (10 cents each) <a href="http://www.ebay.com/itm/Tactile-Push-Button-Switch-Momentary-Tact-Cap-12x12x7-3mm-KeyCaps-Assorted-Kit-/221867170115" target="_blank">sample on ebay</a></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">A few short 22-gauge wires</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">USB charger ($1.10) <a href="http://www.ebay.com/itm/1A-USB-Power-Adapter-Wall-Charger-US-Plug-FOR-Phone-charger-direct-charge-/231982383681" target="_blank">sample on ebay</a></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Scrap wood stand (3" X 8" X 3/4 inch think)</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">3x 1 inch wood screws</span></li>
</ol>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Here are the steps to make your own sign.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Step 1: Calculate the size of sign and spacing of the pixels</span></h3>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi85xkPlssBiNRJOwLi-1oPJ8bCph_SWQDY08L1LQRoGE_fSgJMrzMxUOA3e4us_HpY-aUzAXT_-qlF0ZAYXou9m06DjCUEzyIFCi3NVqIFDIE4BAuvr9o2as127XgjnRcxxpPI2w/s1600/measure-bin-size.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="94" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi85xkPlssBiNRJOwLi-1oPJ8bCph_SWQDY08L1LQRoGE_fSgJMrzMxUOA3e4us_HpY-aUzAXT_-qlF0ZAYXou9m06DjCUEzyIFCi3NVqIFDIE4BAuvr9o2as127XgjnRcxxpPI2w/s320/measure-bin-size.PNG" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">When we do our CoderDojo sessions or when I do maker fairs I take along a plastic bins that are about 20 inches across. I designed the sign to fit in the box, so I made the width 19 inches. To keep the sign proportional to the word "ARDUINO" I needed a sign that was about five inches high.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I wanted to use a 50 pixel LED string so I calculated that most of the letters (except the letter "I") needed to be 7 or 8 pixels each. The I only needed 4 pixels fir the letter "I". So 50 pixels would be fine. You can use longer chains of LEDs if you need to by connecting the DATA OUT to the next string's DATA IN.</span></div>
<h2>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Step 2: Find some scrap diffused white Plexiglas</span></h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6kQv_D1QoLdZoXjNLh_p1gbMxtQawisdqL9DFLN3ZOvccLhp3mRy7-fJhrR31gljyGASSd63b16QAGCQZ5Ym3fjv_TEuaGqZGh8biBPYHMFvtVRLfrF9I2DVeiwXMtCLVlO7XVw/s1600/01-translucent-plexiglass.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6kQv_D1QoLdZoXjNLh_p1gbMxtQawisdqL9DFLN3ZOvccLhp3mRy7-fJhrR31gljyGASSd63b16QAGCQZ5Ym3fjv_TEuaGqZGh8biBPYHMFvtVRLfrF9I2DVeiwXMtCLVlO7XVw/s320/01-translucent-plexiglass.jpg" width="240" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I went to a local surplus store (The AxMan) near my home and found a large piece of 1/4 inch think Plexiglas for $8. I could easily make six signs out of this one piece. You want the translucent kind since it is ideal for diffusing the light.</span><br />
<h2>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Step 3: Cut and Sand the Plexiglas</span></h2>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I cut the Plexiglas on a jig saw and sanded one side with 100 grit sandpaper. I needed to take the gloss out of the Plexiglas so the hot glue would stick to the surface. The front side of the sign can glossy, just the back side needs to be roughened up.</span><br />
<h2>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Step 4: Print the "ARDUINO" sign letters on paper</span></h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieKem7Rjrdahl7UYP_p5t65bXls2-255eUCE9tgEPixQ_gK2062o5LeSjjiZgg1rFXGusmuZjoFdozjrDav9Te5JfMpMH1d69Uq09IaWRyIMic4nVoITIPoJzRfORJXLaU8baygA/s1600/03-print-white-on-black-background.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieKem7Rjrdahl7UYP_p5t65bXls2-255eUCE9tgEPixQ_gK2062o5LeSjjiZgg1rFXGusmuZjoFdozjrDav9Te5JfMpMH1d69Uq09IaWRyIMic4nVoITIPoJzRfORJXLaU8baygA/s320/03-print-white-on-black-background.jpg" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">I used PowerPoint to lay out a two-page sign. I used white letters on a black background and printed it on standard photocopy paper on two pages. A template in PowerPoint of the sign is here:</span><br />
<br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo}
span.s1 {font-variant-ligatures: no-common-ligatures}
</style>
<br />
<div class="p1">
<span style="color: #444444; font-size: x-small;"><a href="https://github.com/dmccreary/moving-rainbow/tree/master/src/displays/signs/Arduino-19-inch-template.pptx?raw=true" target="_blank">https://github.com/dmccreary/moving-rainbow/tree/master/src/displays/signs/Arduino-19-inch-template.pptx?raw=true</a></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Make sure you don't print on thick paper, it will block out too much of the light.</span></div>
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Step 5: Tape letters to the sign and locate LED positions</span></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgB_MR7O86Rj4DqsvWtubeuCjsiJU9P30zs-WZLS4tV8PoJSI-xlpQMW-4cZwCvKZgorQEORH8yjlCOEngfeskxWizJf5fRAgGs6LP8-Th3Ng9DH95JvsjL8pVH6G6Y_gP1A1DUSw/s1600/05-mark-points-on-back.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgB_MR7O86Rj4DqsvWtubeuCjsiJU9P30zs-WZLS4tV8PoJSI-xlpQMW-4cZwCvKZgorQEORH8yjlCOEngfeskxWizJf5fRAgGs6LP8-Th3Ng9DH95JvsjL8pVH6G6Y_gP1A1DUSw/s320/05-mark-points-on-back.jpg" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I then taped the letters to the front of the sign and then turned the sign over. I located each letter and put small marks where each LED would be positioned over the white letters.</span><br />
<h2>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Step 6: Hot glue LEDs in place</span></h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZ491Ci0p_xAKIRFBdX5Vkp2BZHQR9H5y3M6CJmaEldlvWZHZRvlA6yBIhvGA8AOQJYXKrFXV0j5yJpxSfhaA0F1zXmTcHfzImWaDfNYff7batWDAibRZiOFMo2SS9gx6Z8FPsCQ/s1600/06-hot-glue-to-back.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZ491Ci0p_xAKIRFBdX5Vkp2BZHQR9H5y3M6CJmaEldlvWZHZRvlA6yBIhvGA8AOQJYXKrFXV0j5yJpxSfhaA0F1zXmTcHfzImWaDfNYff7batWDAibRZiOFMo2SS9gx6Z8FPsCQ/s320/06-hot-glue-to-back.jpg" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">This was the hardest part of the project. This is also where a little planning and patience pays off. I had to glue the tip of each pixel down onto the Plexiglas so it would stand up. I had to wait until the hot glue hardened before I could go on to the next pixel. I also had to make sure that the last pixel in each letter was near the start of the next letter. My string only had three inches between each pixel so I had to be careful about the order of pixels in each letter.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">If you need more than three inches you may need to cut the wires and solder in some extension wire. Just make sure to keep the colors matching. I also left 3/4 inch of room at the bottom so I had room to mount the sign on the wooden stand.</span><br />
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Step 7: Attach the stand to the Plexiglas</span></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-PCP6uJ6GwR1mVT6jjH_dugWVRpiyz_DksVGVzy1lP9VPII2E2VQQw72auKV3SSYKWRpJ2oWc_0WwWJQctwYqrdmJwnYFWyiJRshsgX_i8irdHlFNG6LvLLbzXMhAYvGl0fLcaQ/s1600/07-attach-base.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-PCP6uJ6GwR1mVT6jjH_dugWVRpiyz_DksVGVzy1lP9VPII2E2VQQw72auKV3SSYKWRpJ2oWc_0WwWJQctwYqrdmJwnYFWyiJRshsgX_i8irdHlFNG6LvLLbzXMhAYvGl0fLcaQ/s320/07-attach-base.jpg" width="240" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Once the LEDs were hot glued onto the Plexiglass I attached the wooden stand to the Plexiglas using three wood screws. I pre-drilled holes so that they were easy to screw in.</span><br />
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Step 8: Mount the breadboard with Arduino to the stand</span></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyoGp1mTNokSQDtL4e-u4_rZIhmI1kW9k9RBdBdGdSTxemoZ1Bo8jgZ2wl8DJLbCcDTtsaiGCcEcBFD8s3onWg4MmYugLgYX5P_Td0lzLL-0AdPqoMW6nF1_-qo-81wVOax7IQ2g/s1600/08-attach-breadboard.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyoGp1mTNokSQDtL4e-u4_rZIhmI1kW9k9RBdBdGdSTxemoZ1Bo8jgZ2wl8DJLbCcDTtsaiGCcEcBFD8s3onWg4MmYugLgYX5P_Td0lzLL-0AdPqoMW6nF1_-qo-81wVOax7IQ2g/s320/08-attach-breadboard.jpg" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Most of the 1/2 size breadboards come with an adhesive backing. I just peeled off the backing and put it directly on the top of the stand. The circuit on the breadboard is pretty simple. The power and ground of the LED strip must be connected to the 5V and GND of the Arduino Nano and the momentary buttons need to pull pins 2 and 3 to ground when they are closed. One button can be used to go to the next pattern and one goes to the previous patterns. Note that I put a drop of hot glue over the connectors to the LED strip. I also soldered solid wire to the ends of the LED wires. Note that in the version I purchased the white is ground, the red is 5v and the green is the DATA IN. Make sure you use the correct end of the strip (not the DATA OUT).</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Here is the circuit I used:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihgu41_NFB-blRysN0Uobia6UFED-RoslnfXpYGfQcerqKDqxd41jjB7XYXta-7NdhBz2jgWOvVOSP84of6t4wJEYkOutn0E228zq0DmzvjhjxLK6pRgXL78BPAcwq7QsIo_yY3g/s1600/breadboard-layout.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="259" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihgu41_NFB-blRysN0Uobia6UFED-RoslnfXpYGfQcerqKDqxd41jjB7XYXta-7NdhBz2jgWOvVOSP84of6t4wJEYkOutn0E228zq0DmzvjhjxLK6pRgXL78BPAcwq7QsIo_yY3g/s320/breadboard-layout.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Note that the buttons are hooked to pins 2 and 3 of the Arduino Nano. These are the interrupt pins. The pins are configured as INPUT_PULLUP so they pull the pins up to 5v when the buttons are OPEN. Pressing the buttons down pulls the pins voltage down to ground and moves the pattern mode to the next or previous pattern. Note that you don't need any pullup resistors when you use these settings.</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Step 9: Load the software on the Arduino</span></h3>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I then plugged in the USB cable and downloaded some sample code to make the LEDs display a nice pattern. Many of these sample programs are on the Moving Rainbow github site here:</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">https://github.com/dmccreary/moving-rainbow</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">The github has many programs and a git-generated book here:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">http://moving-rainbow.readthedocs.io/en/latest/README/</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">You can find the program that allows you to switch between modes here:</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">https://github.com/dmccreary/moving-rainbow/blob/master/src/mini-maker-faire/led-strip-two-buttons/led-strip-two-buttons.ino</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Note that this sample program uses the Adafruit Neopixel library. You can use the Sketch -> Include Library...Manage Libraries... function and just search for "Neopixel" to include the library. There are complete instructions in the online Arduino Moving Rainbow book.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">There are about a dozen color patterns that you can choose. We encourage you to have your students write their own patterns, each within its own subroutine.</span></div>
<div>
<h2>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Step 10: Add a wall adapter</span></h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgETrUWh8A-z4IwaZrTkH20xqXMFBy0TulXOaA7Hq4WLvoT2PPXP7lVKstzL4H1UWjY5VTw-M_sp6LynAERUDaWB-6v1Z77JQZcNDncvK1NJAks-g642SzZvoZ9Xj-q8x3g5lIT9A/s1600/09-power-with-5v-charger.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgETrUWh8A-z4IwaZrTkH20xqXMFBy0TulXOaA7Hq4WLvoT2PPXP7lVKstzL4H1UWjY5VTw-M_sp6LynAERUDaWB-6v1Z77JQZcNDncvK1NJAks-g642SzZvoZ9Xj-q8x3g5lIT9A/s320/09-power-with-5v-charger.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div style="clear: both; text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Here is a video of one of the patterns:</span></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.blogger.com/video.g?token=AD6v5dzjBIZXfJwnKbIiNQAz6WmjiimS3NDw9GZYrMR6Su1EALGQugoCryi7evAwn0kiI5nd2FSfv1o9Is4' class='b-hbp-video b-uploaded' frameborder='0'></iframe><br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">There are a few other ideas:</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ol>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Create a version of the software that continually cycles through each of the modes.</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Hook up a <a href="http://www.ebay.com/itm/High-Quality-HC-SR501-Infrared-PIR-Motion-Sensor-Module-For-Arduino-Raspber-/191990594110" target="_blank">Passive Infrared (PIR) motion sensor</a> to automatically change the mode when there is motion but go into a low-power mode when there is no motion detected. </span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Have students each add their own pattern and alternate between these patterns</span></li>
</ol>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-15935831462403815442015-01-09T20:44:00.003-06:002015-01-11T07:58:09.730-06:00$30 CoderDojo Arduino Robot Version 3<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7VAeO0CguxCk0yAGO_QFyEEOnllEquvUzsKZOJSbMlIio8KWMHetbpOTB03wfO2i2kuANB1hi-QYbC5OAzUJ3CLeoN-DXPudez-jwHstlDSEVHRhAhBDqlUz27o1AAZTS9RZV4Q/s1600/top-big.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7VAeO0CguxCk0yAGO_QFyEEOnllEquvUzsKZOJSbMlIio8KWMHetbpOTB03wfO2i2kuANB1hi-QYbC5OAzUJ3CLeoN-DXPudez-jwHstlDSEVHRhAhBDqlUz27o1AAZTS9RZV4Q/s640/top-big.jpg" /></a></div>CoderDojo Robot Version 3 Top<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfImrHVHrEkj45m7EGI-Ii3VAFjOsdrD7xSxpSs0YQFDfhJNplp7wrJgOr2VLYKobdadpy2LHxvuAgzPYNOmu83zcd8J0aHl5mA91qCEj714ctU_qXVvmAoR00evXBI9bAjRDbig/s1600/bottom-big.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfImrHVHrEkj45m7EGI-Ii3VAFjOsdrD7xSxpSs0YQFDfhJNplp7wrJgOr2VLYKobdadpy2LHxvuAgzPYNOmu83zcd8J0aHl5mA91qCEj714ctU_qXVvmAoR00evXBI9bAjRDbig/s1600/bottom-big.jpg" /></a></div>CoderDojo Robot Version 3 Bottom<br />
<br />
I was happy to get a warm reception last night at our local Arduino.mn user group when we presented version 3 of our $30 Arduino CoderDojo robot. We got great feedback and many request for the bill of materials for people that wanted to try to build their own robot.<br />
<br />
Our goal is to keep kids engaged in math and and science and one of the best ways to do this is to create engaging yet low-cost kits that use light, colors and motion. We place a premium on keeping costs low so that students, can take them home and show their friends. We also want to make the cost effective so that schools, libraries and STEM organizations can purchase them in bulk. This robot fits that mission. <br />
<br />
Just to review, the goal of this kit is to build on the 20 SparkFun Inventors Kit labs. We want to have a low cost, open platform, extensible robot that teaches the following concepts:<br />
<br />
<ol><li>Lab 1: Turn a motor forward and backward using the L293D motor controller.</li>
<li>Lab 2: Change the speed of a one motor using PWM</li>
<li>Lab 3: Moving Forward: Make both motors in the same direction for 1 second and then stops for 1 second.</li>
<li>Lab 4: Turning: Write a function that moves motors in opposite directions for 1 second.</li>
<li>Lab 5: Measuring distance - using an ultrasonic sensor to measure distance.</li>
<li>Lab 6: Display distance - use the LED strip to show the distance to an object in front of the robot.</li>
<li>Lab 7: Avoiding obstacles - move forward till you get close to something in front and then turn.</li>
</ol><br />
Because the Arduino.MN group is full of Arduino makers that want to build their own robots I got lots of request for the hardware I am using. So here it is the Bill of Materials. There are links to eBay however you may find these parts on many other sites. Note that many parts come from China and typically have a 3-5 week delivery time when they come via China Post. However this is the best way to keep the costs down.<br />
<br />
Although the links may not work in the long run, we have also provided some sample keywords to find similar items.<br />
<br />
<table><thead>
<tr> <th>Part</th> <th>Price</th> <th>Keywords</th> <th>Sample Links</th> <th>Notes</th> </tr>
</thead> <tbody>
<tr> <td>Chasis</td> <td>$13.00</td> <td>Arduino Robot Chasis Kit</td> <td><br />
<a href="http://www.ebay.com/itm/2WD-Motor-Smart-Robot-Car-Chassis-Kit-Speed-Encoder-Battery-Box-for-Arduino-1-48-/361025014704">link1</a><br />
<br />
<a href="http://www.ebay.com/itm/2WD-Motor-Smart-Robot-Car-Chassis-Kit-Speed-Encoder-Battery-Box-For-Arduino-1-48-/311172280337">link2</a><br />
<br />
</td> <td>There are many similar kits or you can just get the motors and build your own chassis out of wood. Make sure your kit has a battery pack.</td> </tr>
<tr> <td>Motor Drive</td> <td>$2.33</td> <td>Arduino Motor Drive L293D</td> <td><a href="http://www.ebay.com/itm/Mini-Motor-Drive-Shield-Expansion-Board-L293D-Module-For-Arduino-UNO-MEGA2560-R3-/200982006661">link</a></td> <td>This is a great little board at a great price. The L293D drives up to 600ma. The small DC motors only need about 100ma.</td> </tr>
<tr> <td>Arduino Nano Compatible</td> <td>$4.39</td> <td>Arduino Nano</td> <td><a href="http://www.ebay.com/itm/Arduino-compatible-Nano-V3-0-ATmega328-Mini-USB-Micro-controller-Board-Cable-/251657174814">link</a></td> <td>Note this come with a VERY short USB cable. You may want to pick up a longer one.</td> </tr>
<tr> <td>Ultrasonic distance sensor</td> <td>$1.10</td> <td>Arduino Ultrasonic Sensor HC-SR04</td> <td><a href="http://www.ebay.com/itm/New-Arduino-Ultrasonic-Module-HC-SR04-Distance-Sensor-Measuring-Transducer-HH-/191444060463">link</a></td> <td>Try other distance sensors if you would want.</td> </tr>
<tr> <td>LED Strip</td> <td>$1.10</td> <td>LED RGB Addressable WS2812B 5V</td> <td><a href="http://www.ebay.com/itm/WS2812B-WS2811-RGB-LED-Strip-5M-150-300-Leds-60LED-M-Individual-Addressable-5V-/231105154168">link</a></td> <td>Getting feedback on the ping sensor is key to learning how the work.</td> </tr>
<tr> <td>Main Breadboard</td> <td>$1.74</td> <td>Solderless Breadboard 400 tie points</td> <td><a href="http://www.ebay.com/itm/Mini-Breadboard-Solderless-Protoboard-PCB-Test-Board-400-Points-Contacts-Tie-/261705218882">link</a></td> <td>Look for 10 pcs to save on shipping.</td> </tr>
<tr> <td>Sensor Breadboard</td> <td>$1.00</td> <td>Solderless Breadboard 170 tie points</td> <td><a href="http://www.ebay.com/itm/Mini-Solderless-Prototype-Breadboard-SYB-170-Tie-point-for-Arduino-RANDOM-COLOR-/371025168423">link</a></td> <td>Many colors available.</td> </tr>
<tr> <td>Jumper Wire</td> <td>$1.28</td> <td>dupont wire jumper male female 20cm</td> <td><a href="http://www.ebay.com/itm/40pcs-20cm-Dupont-Male-to-Female-Breadboard-Jumper-Wire-Raspberry-Pi-Arduino-XGT-/121514831974">link</a></td> <td>We only need 6 but they come in handy.</td> </tr>
<tr> <td>On/Off Switch</td> <td>$0.30</td> <td>10PCS mini toggle switch spdt MTS-102</td> <td><a href="http://www.ebay.com/itm/10PCS-Blue-MTS-102-3-Pin-SPDT-ON-ON-6A-125VAC-Mini-2-Position-Toggle-Switches-/261497498956">link</a></td> <td>The packages of 10 that sell for $2.70 are many pages in. The keyword "MTS-102" works!</td> </tr>
</tbody> </table><br />
A few hints about searching for parts on eBay. The search engine is not very good and the sort-by-price rankings are often wrong. Remember to use the "Buy it Now" button. Also watch for the shipping prices. You may have to try many different combinations of keywords to find the right component and the right quantity.<br />
<br />
If you are not finding a low-priced item, just keep scrolling. The right price might be buried 10 pages in and eBay's "sort by price" seldom works and had no cost or price per item.<br />
<br />
I also tend to purchase generic components like jumpers and wire in batches of ten or more since I know I can reuse them and I get breaks on shipping costs.<br />
<br />
As pointed out in the meeting, the Arduino Nano uses a driver that does not work on the latest version of the Mac OS. We hope they fix the driver problems soon.<br />
<br />
My Powerpoint/KeyNote slides are also available if you would like them.<br />
<br />
The sample code is on github:<br />
<a href="https://github.com/dmccreary/moving-rainbow/blob/master/src/robot/robot_ping_led/robot_ping_led/robot_ping_led.ino">Github Code</a><br />
<br />
Many thanks to Doug (who has been really helpful helping me find low-cost parts on eBay) and Gerd, who has been a great mentor to me and Eric Palmer, who keeps me motivated.Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-10166625432505172522014-11-08T05:49:00.002-06:002014-11-08T05:49:47.073-06:00Robot Kits for CoderDojo Labs<span style="font-family: Arial, Helvetica, sans-serif;">For a few months our CoderDojo project in Minnesota had access to several <a href="http://www.finchrobot.com/" target="_blank">finch robots</a>. These were wonderful introductory systems with many different programming languages (C, JavaScript, Python, Snap, etc.), however they were expensive (about $100), and lacked extensibility. There was no way to enhance the robots with new sensors or change the motors. They were theather by the USB which limited their motion. And in an ironic twist of fate, we found the finch robots simply can't <a href="http://en.wikipedia.org/wiki/Darwin's_finches" target="_blank">evolve</a> with our kids as they learn and grow.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">The SparkFun Inventor's Kit that we use in our current Arduino labs do have the basics for robot control. There are labs for controlling a servo and a useful one-way rotation motor lab. However the kit does not have a basic H-bridge lab so I often bring some basic L293D chips and have the kids google "L239D Arduino" and show them <a href="http://www.instructables.com/id/Control-your-motors-with-L293D-and-Arduino/" target="_blank">this page</a> which has a nice lab on the Instructables site. In this lab they learn how to make a motor turn both forward and backward, a key skill for controlling the robot motion.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Working with kids, I know how important it is to keep them motivated to lean more and find the hooks that keep them interested and want to come back for more. So I have taken it upon myself to try to replace the finch robots with an more open and cost effective platform. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Here are some of my selection criteria.</span><br />
<ol>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Extend the current CoderDojo Arduino Labs</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Use the same Arduino IDE to program the robot (more on this later)</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Keep the total price under $100 and under $75 if possible</span></li>
</ol>
<span style="font-family: Arial, Helvetica, sans-serif;">In doing my research I came across the book "<a href="http://shop.oreilly.com/product/0636920028024.do" target="_blank">Make an Arduino-Controlled Robot</a>" by Michael Margolis. This book which has a list price of around $25 (Amazon has it for $17) has most of the things that we are looking for. Low cost <b>and </b>extensibility. The book is not perfect (see the <a href="http://support.oreilly.com/oreilly/searches?query=Arduino-Controlled%20Robot" target="_blank">support questions</a>) and does have examples from an older version of a motor shield from Adafruit. We hope a revised version comes out soon. However, the book does have some good ideas and we have a good general instructors guide to start with.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">The next step is to find a basic robot kit for a reasonable price. I came across a kit on ebay called <a href="http://www.ebay.com/itm/321443340835?_trksid=p2059210.m2749.l2649" target="_blank">UNO R3 2WD smarter Car Speed Encoder L298N HC-SR04 Sensor Kit Robot for Arduino</a> that lists for $47.99:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Here are the parts that the kit comes with:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">1 Car chassis</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">2 x Gear motor, tire, encoders</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">1 Universal wheel</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">1 Battery box (batteries not included)</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">1 BUONO UNO R3 </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">1 L298N Motor Driver </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">1 HC-SR04 Ultrasonic Ranging Sensor </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">1 Sensor Kit V05 </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">1 female to female cable</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">1 male to female cable</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">1 male to male cable</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I should also note that the motor drive is a bit different then the Adafruit driver. The L297N we can get for about $5.00 on eBay. The Adafruit version 2 motor shield is about $20.00 + shipping.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I have one of these kits on order and I will see how it goes to get it working. I think the key is that the kits are reasonably priced and barring any barring any integration issues we should be able to use the parts to get us started on the basic forward, reverse, turn left and turn right labs. Getting all the sensors the to work will still require some work on our part. And if we get approval to purchase the kits I would want to get at least five of the mentors some training on how to use them. We will also need to purchase four, AA batteries for each of the robots.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I should also mention that I asked everyone at the Arduino.mn meeting for suggestion and I got many of them. There are many more options for <a href="http://www.robotshop.com/en/robot-kits.html" target="_blank">kits</a> from sites like <a href="http://www.dfrobot.com/" target="_blank">DFRobot</a>. There are literally 100s of kits and options in the $100 to $200 range. </span><span style="font-family: Arial, Helvetica, sans-serif;">The basic </span><a href="http://www.radioshack.com/arduino-robot-us-plug/2760355.html" style="font-family: Arial, Helvetica, sans-serif;" target="_blank">Arduino Robot</a><span style="font-family: Arial, Helvetica, sans-serif;"> from Radio Shack runs for $274.00. However none of them really seem to have better value for parts then the basic ebay kit listed above.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Many people suggested that kids would love to "drive" their robots with a bluetooth remote control. One person told me that the IR remotes were too hard to use since you had to stand in front of the robot to get them to work. These are all options that we can evolve into once we have the robot platforms picked out.</span><br />
<h2>
<span style="font-family: Arial, Helvetica, sans-serif;">IDEs, Robots and the Limitations of the Arduino</span></h2>
<span style="font-family: Arial, Helvetica, sans-serif;">I should also mention that many of the limitations on getting the robots to work is the noise reduction code in the sensors. Sensors are inherently noisy and the kids really need to master the art of cleaning up the noise in their code to get the robots to perform.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">The Arduino Integrated Development Environment (IDE) has a limited ability for the kids to see the actual signal from the sensors. Which is sad. Putting a simple graph option in the serial log should be about two pages of Java code. The students can print the values of the sensors on the serial port, but their is no way to graph them and see the noise levels. The lack of plugins for the IDE really make it limited, something the Eclipse team learned many years ago.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">My friend, Eric Palmer, is also looking into a better "Universal IDE" called, wait for it..<a href="http://uecide.org/" target="_blank">.The Universal Embedded Computing IDE</a> or the UECIDE. This IDE will work with both the basic inexpensive Arduino chips as well as the newer and MUCH more powerful mbed chips. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Since the Arduino chips really are somewhat limited in terms of memory and speed, many of the advanced robotics labs may need to leave the safe world of Arduino and use one of the much more powerful and low-cost <a href="http://en.wikipedia.org/wiki/Mbed_microcontroller" target="_blank">MBED</a> chips. The ARM chips are 6x faster and have 16x the memory as well as options for built-in bluetooth and WiFi. Here is a nice article on a <a href="http://blog.mostlyrobots.net/2013/07/08/avc-switching-from-arduino-to-mbed/" target="_blank">comparison</a>.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Eric has also mentioned that he is building a popular "<a href="http://www.sainsmart.com/sainsmart-balancing-robot-kit.html" target="_blank">self balancing robot</a>" using the mbed processor. These robots are great ways to teach robot control and dynamic feedback. They demonstrate how to apply both derivatives and integral that we teach in typical freshman Calculus classes. This is something that our little Arduino may not able to do. If we want to do self balancing robots we need to take our Arduino training wheels off (literally) and move to a better IDE and a faster board. However our platforms should still be useable with this software and hardware.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Although this IDE is much more "Eclipse" like, I did get it to run the blink lab in about 5 minutes. So the setup is not too much more difficult.</span><br />
<div>
<br /></div>
Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-1580022127539450372014-10-26T12:34:00.001-05:002014-10-26T12:44:22.000-05:00Internet of Things Hackday Trip Report<span style="font-family: Arial, Helvetica, sans-serif;">Last week I attended the <a href="http://iothackday.mn/" target="_blank">Internet-of-Things Hackfest</a> at <a href="http://www.minnetronix.com/">Minnetronix</a> is the Twin Cities.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWqISWw2M006YdYYIDN1liGbr4UMkJrfjNmZr0tY_Te5HbHtbWh-y0Avr7cIBCLJpttMoPLgipc3pYxL6y8gk85jhSDghBbyC-mKoan7LsyjMy5fMJ-3ETfmFbTUX_xycjE7W-hA/s1600/Dan-teaching-arduino.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: Arial, Helvetica, sans-serif;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWqISWw2M006YdYYIDN1liGbr4UMkJrfjNmZr0tY_Te5HbHtbWh-y0Avr7cIBCLJpttMoPLgipc3pYxL6y8gk85jhSDghBbyC-mKoan7LsyjMy5fMJ-3ETfmFbTUX_xycjE7W-hA/s1600/Dan-teaching-arduino.png" height="299" width="320" /></span></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b><span style="font-family: Arial, Helvetica, sans-serif;">Dan McCreary working with a student on the Moving Rainbow Arduino Kit</span></b></div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Here is a quick summary of some of the things I learned last week:</span><br />
<ol>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Students LOVED the new <a href="http://datadictionary.blogspot.com/2014/08/new-arduino-moving-rainbow-lab-for-kids.html" target="_blank">Moving Rainbow</a> kits I built. Although some designs worked better than others.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">The <a href="http://spark.io/">spark.io</a> device is wonderful but still lacks some features that would make it easy for students to use in classrooms.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Getting IoT remote devices to work requires specialized skills (like C parsers) that many of us don't have.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">When students get their own ideas for projects they get laser focus</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Hackdays are a lot of fun! They are great ways to meet people and learn from others.</span></li>
</ol>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I had two roles at the hackfest. I helped setup and run the "Kids Room" for the hackfest and I did a (very) little work in the <a href="http://lumiere.lighting/" target="_blank">Lumière lighting</a> team.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">For the Lumière team I mostly helped a few people to learn how to solder wires the LED strips and use heat shrink to secure the wires. The other two guys on my team (Daniel Feldman and Alan) had much better programming skills and experience with other platforms like the Raspberry Pi.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I did spend many hours working on building an XForms, REST, XQuery application for the spark.io. And although I did get it working (<a href="https://www.youtube.com/watch?v=6cy10dsFKkM" target="_blank">video</a>), it still needs a lot of polish to allow the commands to be flexible. I also did get a few new patterns working:</span><br />
<ol>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Larson Scanner (Cylon)</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Random colors</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Candle flicker (simulate the light flickering pattern of real candles)</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Up-down patterns</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Swipe (redraw colors one pixel at a time)</span></li>
</ol>
<span style="font-family: Arial, Helvetica, sans-serif;">Many of these still need to be parameterized with arguments such as color and speed. However doing a general interface still needs work.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">To get the Internet-of-Things (IoT) moving rainbow kits to work we need to send a specific "change color pattern" command to each device. Since the spark.io interface only accepts a simple string, we need to parse the strings into a pattern of colors and motion. However none of use knew how to use the strtok_r() function so we struggled a bit. Daniel did a great job under pressure getting one command parser working on the spark.io, however we will need to refine it a bit more over time.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">The ideal long-term solution is to develop a small "Addressable LED Strip Markup Language" as a domain-specific command language and write a parser for it in C that would run on the Spark.io. This is a non-trivial problem for people that don't write C every day. However I hope to spend some time thinking about the pattern name, color name, delay period syntax and perhaps coming up with a small BNF grammar.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<b><span style="font-family: Arial, Helvetica, sans-serif;">Moving Rainbow Findings</span></b><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Now I want to review some of the findings around a low-cost Arduino kits for students.</span><br />
<b><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></b>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXU-KxqzvGRTppitAa9kNWLyggMxLNZXKRvs-kNhn6F9Ty0w3rg0dFiAU4uI7tmdqVlJQhj2UW6XjT2rLYre-03dBC_RLbboUornEylVJqzk2EIvc6xzuCt1-7dg2lvHx4LgwBLw/s1600/kids-arduino-hackday.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXU-KxqzvGRTppitAa9kNWLyggMxLNZXKRvs-kNhn6F9Ty0w3rg0dFiAU4uI7tmdqVlJQhj2UW6XjT2rLYre-03dBC_RLbboUornEylVJqzk2EIvc6xzuCt1-7dg2lvHx4LgwBLw/s1600/kids-arduino-hackday.jpg" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b><span style="font-family: Arial, Helvetica, sans-serif;">One of the dongle-style Moving Rainbow designs connected to an FTDI programmer</span></b></div>
<b><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></b>
<span style="font-family: Arial, Helvetica, sans-serif;">My primary interest is to understand what can we do to get students involved in STEM and to keep them on the strong math and science tracks in high-school and college. Each time I engage with students I look for things that get them excited and want them to return for more.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">My vision is to help design low costs kits that students could take home and show their friends. These kits should be inexpensive enough that schools, libraries, and park buildings could have them for checkout, just like any library book. Imagine a bookshelf at the library that contained 100 different electronic kits, each with specific learning goals in mind.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">My experience has shown me that with some mentoring and some good electronics kits that many </span><span style="font-family: Arial, Helvetica, sans-serif;">students </span><span style="font-family: Arial, Helvetica, sans-serif;">can use internet resources to do a lot of work on their own. However some students need a bit more encouragement then others.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">This was one of the first times I had my newest collection of my "Moving Rainbow" kits on display. When the </span><span style="font-family: Arial, Helvetica, sans-serif;">students </span><span style="font-family: Arial, Helvetica, sans-serif;">came into the room I encouraged them to pick up each kit and play with them. Each of them has slightly different designs, packaging and switches. When they flick the power switch on the 12-segment RGB LED lights came on and some of the kits had nobs and sensors that change the LED patterns displayed. I tried to pay special attention to each of the students and they came into the room. I noted what kits they picked up and what features they were interested in.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">One of the things I learned is that there was little interest in the small-versions of the kits. These were the ones that did not have room for the 12 pixels within the box. These had a "dongle" design where the LED strip was sticking outside the boxes. These designs pretty much failed, and looking back I can see why. My reason to try them was that the smaller boxes were less expensive. The larger boxes cost $5 or $6. However, one of the students wanted to purchase the "dongle" designs.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">The other thing to remember is that the logistics of getting the Arduino drivers working on both Windows and Apple systems is very time consuming. Much of the first hour was getting the Arduino drivers working and the FTDI drivers (required for the mini-pros) installed.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Several of the </span><span style="font-family: Arial, Helvetica, sans-serif;">students </span><span style="font-family: Arial, Helvetica, sans-serif;">did a "color wheel" lab where they had to mix the red, green and blue values together to make purple, yellow and orange colors appear on the LED strip. The look on their faces when the strip turned either the right or wrong colors was priceless. These were fantastic teaching moments! On my TODO list is a "cheat sheet" for the color functions with pictures of color wheels. This guide would start with functions to set a pixel to a specific RGB value and draw colors within for loops.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">The moving color labs was a bit harder. I need to continue to find good simple examples of lights moving up and down and get them pre-installed in the Arduino Examples area. This is a great way to teach for loops and if/then/else logic. As my sister and brother have told me, teacher prep makes a huge difference in students understand and lowering frustration levels.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">One incident was that the students really <b>loved</b> the idea of creating a small wearable designs. I had my "<a href="http://www.instructables.com/id/Arduino-and-battery-pack-in-Altoids-tins/" target="_blank">Altoids</a>" example necklace as a demo to show them. A few of them wanted to make Halloween costumes out of them. And they were VERY motivated once they realized they could create their own costumes out of them. At our final presentation to the entire group, two of them came to the front of the group to show their creations. One of the students specifically thanked me in front of the entire group. I wish all the students were this polite!</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">One idea is to have a "Arduino wearables" hackday around the same time next year. This would require a lot of planning and some volunteer work by people with sewing machines.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">What this taught me was that once students get their <b>own</b> idea of a creation it is like a fire-bolt of energy gets lit in them. Their distractions disappear and they become laser focused on their task. They seek whatever resources they can get to reach their goals. Helping each student find their project is what these hackfests is all about!</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I also met a few other people that might help us lower the costs of the packaging and find lower costs of the Moving Rainbow kits. I still have more work to do to learn about placing larger quality orders for components and packaging on volume discount sites such as alibaba.com.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I want to reach out and thank everyone that made the meetup possible. The sponsors and the people at Minnetronix should get a special thanks!</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-38331727199966907632014-10-13T10:35:00.000-05:002014-10-13T11:57:45.561-05:00Motors for Arduino Labs<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">This week I went through many of the SparkFun Inventor's Kits that we use in our CoderDojo Arduino labs and realized that the kids have been loving our little DC mini-motors to death. As you can see in Figure 1 over half of the wires had broken off!</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOdFg8fvSCquFkq91g9lbvo0IXBiYK8H99UnRgNjLWcrmSdEmqk8Wqq6mR4ghG5Zsd20ah_kXHLKdZ7uESIybnNmcuC36Hhm56RIrQKzPEByUwaVxtADOC9DaggWooWLyEP4WH4g/s1600/motors-before.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOdFg8fvSCquFkq91g9lbvo0IXBiYK8H99UnRgNjLWcrmSdEmqk8Wqq6mR4ghG5Zsd20ah_kXHLKdZ7uESIybnNmcuC36Hhm56RIrQKzPEByUwaVxtADOC9DaggWooWLyEP4WH4g/s1600/motors-before.png" height="170" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Figure 1: DC Motors with most connector broken off</b></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The ones supplied in the SparkFun Inventors kits have small delicate wires on them that break off easily. Since the motor circuit lab is a popular lab we need reliable DC motors!</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; white-space: pre-wrap;">It was pretty easy to solder the wires back on. However, I wanted to make sure they didn't get pulled off again. So I have replaced the old thin wires with some thinker 22 gauge stranded wire. I then use heat shrink and some cable ties to bind the wires to the case of the motors. You can see the heat shrink just before I applied the heat in Figure 2.</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; white-space: pre-wrap;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOfeKlMONTazO6TUp40YA33t3wWWtcFkgz4EWsOQEvnOft0QfcOyLcNp8ycczvIlB65JMnph824Ysybn-fCEIQyEUcwyGXuzoI3EfKjLH5lBL_-j2e5A5sBrJdm995M7iVVUfO0Q/s1600/motor-with-heat-shrink.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOfeKlMONTazO6TUp40YA33t3wWWtcFkgz4EWsOQEvnOft0QfcOyLcNp8ycczvIlB65JMnph824Ysybn-fCEIQyEUcwyGXuzoI3EfKjLH5lBL_-j2e5A5sBrJdm995M7iVVUfO0Q/s1600/motor-with-heat-shrink.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Figure 2: DC motors with heat shrink</b></span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b><br /></b></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Next I used some small cable ties (see Figure 3) to firmly secure the wires to the motors. So if the kids pick up the motor by the wires it will not strain the solder joints.</span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJV_voA2G_IozSipt3XywAroTeYXWcS5HlOFHIb5CQDaofn52paEeppoSfQV1Ccqu2si4c0IPlPuZH8R7eCRw6vETq6LfoeQOrS5w31GRWD6NMUmEz6UFmWUGdExipEfwnE8SkxQ/s1600/motor-with-cable-tie.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJV_voA2G_IozSipt3XywAroTeYXWcS5HlOFHIb5CQDaofn52paEeppoSfQV1Ccqu2si4c0IPlPuZH8R7eCRw6vETq6LfoeQOrS5w31GRWD6NMUmEz6UFmWUGdExipEfwnE8SkxQ/s1600/motor-with-cable-tie.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b style="font-family: Arial, Helvetica, sans-serif;">Figure 3: DC with cable tie</b></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; white-space: pre-wrap;"><br /></span></div>
<h3 style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;">Viewing Motor Direction (Optional)</span></span></h3>
<div>
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;">I also realized that some of the motors don't have gears on them so it is difficult to tell what direction they are spinning. In our labs (not in the Sparkfun Inventors Kit) we need to show both clockwise and counterclockwise directions in the labs. The direction of rotation is used in the motor labs that use the motor driver <a href="https://en.wikipedia.org/wiki/H_bridge">H-bridge</a> chips like the popular L293D. So I have added a small drop of hot-glue on the end of the motor </span><span style="font-size: 15px; line-height: 17.25px; white-space: pre-wrap;">spindle</span><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"> and then added a <a href="https://en.wikipedia.org/wiki/Golden_spiral">Golden Spiral</a> sticker to help the students see the direction of rotation. I also put a ring of hot-blue around the base and then used a felt pad so that the motor could stand upright.</span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh42lqVoMmRaZ816D7sPiYviLkgNQ_pSQKQ30lgrJSHDasQVdOZ6hyphenhyphen1T_cTww2mAjVCdc5_ZX1dgSsOGHOYogtsPq5vQRDjoSmppm7SnpPQqNs2OlvJNCqBnEwhaG5FcIy9qQJZiw/s1600/motor-with-golden-spirial.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh42lqVoMmRaZ816D7sPiYviLkgNQ_pSQKQ30lgrJSHDasQVdOZ6hyphenhyphen1T_cTww2mAjVCdc5_ZX1dgSsOGHOYogtsPq5vQRDjoSmppm7SnpPQqNs2OlvJNCqBnEwhaG5FcIy9qQJZiw/s1600/motor-with-golden-spirial.png" height="320" width="265" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b><span style="font-family: Arial, Helvetica, sans-serif;">Figure 4: Motor on base with Golden Spiral label on motor spindle. The wires are multi-stranded 22 gauge but have a 1/2 inch solid gage at each end to work with the breadboard.</span></b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;">Figure 5 has the artwork from the Golden Spiral Label. This is one of may </span><span style="font-size: 15px; line-height: 17.25px; white-space: pre-wrap;">favorite designs.</span></span></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1cumr0SxoxU6HrfDNwvIdJdesuvTSjowevDdnFlOGBK5yXkx8yXnEQ9Snm_iygAr0oWfdc1woeVgZWnWeCtvBK9T36tx91Dy7ROPlrYT0CR3VVuHMS933QtbhFwFfINYL11EENQ/s1600/golden-spiral.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1cumr0SxoxU6HrfDNwvIdJdesuvTSjowevDdnFlOGBK5yXkx8yXnEQ9Snm_iygAr0oWfdc1woeVgZWnWeCtvBK9T36tx91Dy7ROPlrYT0CR3VVuHMS933QtbhFwFfINYL11EENQ/s1600/golden-spiral.png" height="293" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b style="font-family: Arial, Helvetica, sans-serif;">Figure 5: Golden Spiral Label to indicate rotation</b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;">As an aside, some of you may know that spirals are painted on jet engines like in Figure 6 to indicate that the engine is rotating and give the viewer some indication of direction. </span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXAa3LasP7ETEmhAnYk4IV3i7AWqNbgU8YIrI9AvBIhrS1EcZACQH8bTTnd292t5Si62Fi20QZdMXhAQQnU_h3mwQFo4cxLJyCT37YpkutKRsBNaG72m2RTJ0YBGnBRQmvzano3g/s1600/jet-engine-spiral.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXAa3LasP7ETEmhAnYk4IV3i7AWqNbgU8YIrI9AvBIhrS1EcZACQH8bTTnd292t5Si62Fi20QZdMXhAQQnU_h3mwQFo4cxLJyCT37YpkutKRsBNaG72m2RTJ0YBGnBRQmvzano3g/s1600/jet-engine-spiral.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b style="font-family: Arial, Helvetica, sans-serif;">Figure 6: </b><b style="font-family: Arial, Helvetica, sans-serif;">Golden Spiral </b><b style="font-family: Arial, Helvetica, sans-serif;">Graphic painted on jet engine to show motion</b></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;">A discussion of the golden spiral is also a change to discuss the </span><span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;">beauty</span><span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;"> of mathematics. A PDF file that can be used print out the designs on sheet of labels is </span><a href="http://danmccreary.com/electronics/motors/golden-spiral-stickers.pdf" style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;">here</a><span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;">. Let me know if you want the source PPT file. I am not artist and others might have a better spiral design. An SVG image might be a bit smoother.</span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;">So some of you might be thinking that this is a lot of detail for a just one lab! You would be right! However, I think that the care and attention to detail we give each of these labs help the kids get quickly to the next level. This principal helps us apply the theory of </span><span style="font-size: 15px; line-height: 17.25px; white-space: pre-wrap;"><a href="http://epltt.coe.uga.edu/?title=Piaget%27s_Constructivism">Constructivism</a> to our labs</span><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;">. Once kids know they can change motor direction directly from within their Arduino code they can then make the logical next step to seeing how they can turn a robot car direction by having one wheel motor go backwards. Each of these are small logical learning steps we need to keep kids coming back to our STEM classes.</span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;">In a busy </span><span style="font-size: 15px; line-height: 17.25px; white-space: pre-wrap;">noisy</span><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"> class with 20 students jumping around we don't want to have to search through 20 kits looking for a motor that has the wires attached. This takes time away from our learning objectives.</span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;">With a good solid supply of motors that will last through a year of labs like in Figure 7 </span></span><span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;">you will be </span><span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;">ready </span><span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;">to help all your students get to the really fun parts - building their own Arduino powered robot!</span><span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;"> </span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0Gn0UKOf5Kzg-_GoBg99cVoE8Olvx1NFP9uTbmDESh8yCMNCAwgdjFFaWuiUIkSIQxdw8YhTQASKNaFnZQrzIy3toPS1bOXZk0gd7HsNwNb6lifWX0CTSgNcF2ypRGn1suiO3Nw/s1600/motors-after.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0Gn0UKOf5Kzg-_GoBg99cVoE8Olvx1NFP9uTbmDESh8yCMNCAwgdjFFaWuiUIkSIQxdw8YhTQASKNaFnZQrzIy3toPS1bOXZk0gd7HsNwNb6lifWX0CTSgNcF2ypRGn1suiO3Nw/s1600/motors-after.png" height="179" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Figure 7: Motors ready for the kids!</b></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;"> We will hear more about this and the learning steps to building robots in future blog posts.</span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;"><br /></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;"><b>References</b></span></div>
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
</div>
<ul>
<li><span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;"><a href="https://www.sparkfun.com/products/12001">SparkFun Inventors Kit </a>($99.00) </span></li>
<li><span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;"><a href="https://www.sparkfun.com/products/11696">Replacement Motors on SparkFun</a> ($1.95 + shipping) <b>with</b> jumper pins. </span></li>
<li><span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;">Replacement Motors on </span><a href="http://www.ebay.com/itm/5-Pieces-DC-3-6V-3Volt-to-6Volt-Operation-13000RPM-Mini-Motor-Replacement-/311008655229" style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;">ebay</a><span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;"> (5 for $7.85 - no jumper pins. They usually take two weeks for shipping)</span></li>
<li><span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;">Heat shrink tubing on </span><a href="http://www.ebay.com/itm/280-Pcs-1-8-Polyolefin-2-1-Halogen-Free-Heat-Shrink-Tubing-8-Sizes-5-Color-Box-/330922684315" style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;">ebay</a><span style="font-family: Arial; font-size: 15px; line-height: 17.25px; white-space: pre-wrap;"> (280 pieces for $8.79 with multiple sizes and a nice box)</span></li>
</ul>
<br />
<div dir="ltr" style="margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial;"><span style="font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></span></div>
Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com4tag:blogger.com,1999:blog-13546516.post-744046496878602782014-10-09T23:51:00.000-05:002014-10-10T14:36:18.049-05:00Spark.io Core Evaluation<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">Last week at the Minnesota Arduino meeting I met the team from </span><a href="https://www.spark.io/" style="line-height: 1.15; text-decoration: none;"><span style="color: #1155cc; font-family: Arial; font-size: 15px; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Spark.io</span></a><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">. They make a low cost (under $15 in quantity) device that you can program like an Arduino, but it has a full WiFi stack running on it and works with an optional cloud-based service. They gave me (free) a device to evaluate. So here is a first look.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The device came in a small box with a half breadboard and a USB connector. You plug it in and it starts flashing a color that indicates it is "listening for a WiFi login". If I had a smartphone it would have been easy to give it my WiFi ssid and password, however I did figure out how to give it the credentials using a Windows serial port (putty). The serial port is not a full unix-like shell, but it got the job done.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Once it got on the local WiFi I went to the spark.io web site and "claimed" my device by putting in its device ID (which I also got via the Windows serial interface). I could then use a web-based IDE to download the standard Arduino "Blink" app. I also found the NeoPixel library and ported over some of my Moving Rainbow demos. Here is the spark core running one of the rainbow patterns:</span><br />
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZCu3w_K8_cl4-UYceIhdOEQ0muuWUZ8dGZUWEmUGmxL-CLlylQkam-jy9DLet-nAN_0kfbzUO3n05sx88_sd1bQf0IVyC9pKhMJeU94BYW-G5uybQ1FkvIqN3mApbUL_egjiExQ/s1600/rainbow-on-spark.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZCu3w_K8_cl4-UYceIhdOEQ0muuWUZ8dGZUWEmUGmxL-CLlylQkam-jy9DLet-nAN_0kfbzUO3n05sx88_sd1bQf0IVyC9pKhMJeU94BYW-G5uybQ1FkvIqN3mApbUL_egjiExQ/s1600/rainbow-on-spark.png" height="191" width="320" /></a></div>
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">It is interesting to note that you can't use many Arduino Libraries alone. Someone needs to port them. However I think that most of the common libraries that students would need are there.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">What I really liked about the device is that the cloud interface give each device a </span><a href="http://docs.spark.io/api/" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">REST interface</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">. For example if I put the following into my browser (or use the UNIX curl), I can get my device status:</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://api.spark.io/v1/devices?access_token=1234%E2%80%A6" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">https://api.spark.io/v1/devices?access_token=1234…</span></a></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">We can get the status of our device in a JSON format:</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">[</span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> {</span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> "id": "1234...",</span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> "name": "dan-test-1",</span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> "last_app": null,</span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> "last_heard": "2014-10-10T04:19:08.528Z",</span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> "connected": true</span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> }</span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: 'Courier New'; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">]</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">If you have a smartphone there is a nice app you can use to setup the spark core (wifi and password) and do some basic programming. However I also want to make sure that I can figure out how to make the spark core work in a teaching setting similar to the CoderDojo meetings. This is where kids come in with Windows PCs and hook up the Arduino to the USB and fire up the Arduino IDE. I also think that schools may purchase Chromebooks that they also want to use in the labs. So I still have a bit more learning to do to see if I can create a full Internet-of-things lab that kids with a low-cost Windows PC can really use.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">I like the fact that the spark.io system seems pretty open and all the code is on github. The ARM processor also is a LOT more powerful than the Arduino controller. The cost is also very low when you consider that a </span><a href="https://www.sparkfun.com/products/11287" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">WiFi Arduino shield</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> alone is $90 which is 3x more expensive than the entire device. The spark.io Internet of Things "</span><a href="https://www.spark.io/dev-kits" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">starter kit</span></a><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">" is currently $39.00. They have another version with many sensors for $99.00 which seems like a good deal.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">My initial impressions are that if you have a Linux or Mac and can run node.js from the command line that you should have very few problems. That is clearly their target developer audience. However many kids don't have access to a smartphone and a $1,000 Mac. However, having a low cost Windows computer and using an simple Eclipse-like IDE is still something I hope we can have in the future.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">I have started to create a simple web-front end and have some basic unit tests running. I am using Bootstrap 3 and eXist (my favorite NoSQL database) with a few simple XQuery functions to build the URLs. My initial tests allow you to store the device ID and Access codes in a config file and I have a set of functions that convert the JSON responses into pretty HTML pages. Next I want to be able to change the colors and light patterns using a web form. I hope that I have something to demo for the Luminarie team at the hackfest a week from Saturday!</span></div>
<br />
<span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">Overall rating 5 out of 5 stars!</span>Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-89239809526738502932014-08-30T13:17:00.000-05:002014-08-30T13:19:08.461-05:00New Arduino Moving Rainbow Lab for Kids for Under $10<span style="font-family: Arial, Helvetica, sans-serif;">After participating as a mentor in the local Twin Cities <a href="http://www.coderdojotc.org/">CoderDojo </a> in Minneapolis at the University of Minnesota I found that there are two things that kids love: <b>motion </b>and <b>color</b>. My prior experience with kids is that they love to take projects <b>home with them</b> to show their friends. So here is the question - could we design an Arduino kit that kids could come to Maker events and actually take a working project home that has both color and motion?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;">At our local CoderDojo we wisely use the <a href="https://www.sparkfun.com/products/12001">SparkFun Inventor's Kit</a>. If you have not seen this I encourage you to take a look. Lots of nice components and a great <a href="https://www.sparkfun.com/tutorial/AIK/ARDX-EG-SPAR-WEB.pdf">guide</a> (PDF). Many people undervalue the amount of work going into the guides for these kits. Good writing is hard to find, are rarely included in many lower-cost projects. However, since the cost of full Inventors Kit is still around $100 (without shipping) it is beyond the budget of most Maker events to let kids take these home. So here is my answer: lets find a way to use the low-cost Arduino Compatible <a href="http://arduino.cc/en/Main/ArduinoBoardProMini">Pro Mini</a> combined with a short strip of addressable LEDs in a kit that we could sell for around $10.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;">Here is a picture of my initial design:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinfq7bHuUDQiY-v4APkeCSG3IZI55IiQ0yFLmx1MdzwFF09rj05Bs9Bc8WRKRsUbPRBnHcm0NwA3-0-4JtySupn_f60Jquvpi-k5AppzrEnL06M3BID_TvPEGWeKSctlmoToDJ0A/s1600/arduino-rainbow-lab.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinfq7bHuUDQiY-v4APkeCSG3IZI55IiQ0yFLmx1MdzwFF09rj05Bs9Bc8WRKRsUbPRBnHcm0NwA3-0-4JtySupn_f60Jquvpi-k5AppzrEnL06M3BID_TvPEGWeKSctlmoToDJ0A/s1600/arduino-rainbow-lab.jpg" height="212" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b><span style="font-family: Arial, Helvetica, sans-serif;">Arduino Moving Rainbow Kit</span></b></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span></div>
<span style="font-family: Arial, Helvetica, sans-serif;">I think this kit is perfect for kids since we have the wonderful colors of these amazing individually addressable LEDs as well as the ability to create "motion" by programs such as running lights. The total cost of this should be around $10, depending on the options we use.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span><span style="font-family: Arial, Helvetica, sans-serif;">Now lets go through the components to verify that we can do this in the $10 price range.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;">Here is a place on <a href="http://www.ebay.com/itm/Hot-Pro-Mini-Atmega328-5V-16M-Micro-controller-Board-for-Arduino-Compatible-DIY-/291161114258">eBay </a>that sells the Pro Mini Arduino Compatible for around $2.00. Although I suspect that we could get them in quantity for less. Note that these do not have a USB port on them. More about that later.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;">Here is an <a href="http://www.ebay.com/itm/1M-60LEDs-M-WS2811-Led-Digital-strip-light-Dream-RGB-Individually-Addressable-5V-/121136482420">addressable LED strip with 60 LEDs</a> that we can find on eBay for around $17.00. You can cut these up into 6 strips of 10 LEDs each which gives you a price per kit of around $2.80. What is wonderful about these LED strips is that they only take two power pins and a single data pin to hook up! I use the standard WS2811B which you can program with many libraries. I have some sample code on <a href="https://gist.github.com/dmccreary/11252961">github</a> that uses the Adafruit NeoPixel libraries, however I have found other libraries work well also. Keeping wiring simple is important for kids that don't yet have the fine motor skills (and patience) to wire up complex projects.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;">Now the last two components are the<a href="http://www.ebay.com/itm/10x-170-Tie-points-Color-Mini-Solderless-Prototype-Breadboard-for-Arduino-Shield-/281210861472"> mini soderless breadboad</a> (around $.70 in quantity), a switch and a power source. In the kit in the photo above I am using a 3.7volt Lithium Poly battery taken from an old RC helicopter that no longer works. We could also use an external <a href="http://www.ebay.com/itm/10PCS-Lead-Cable-Off-Switch-Battery-Holders-Box-Storage-Case-AA-2A-/261524247424">3 AA or AAA battery holder for around $1</a>.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;">I also added a power switch and a <a href="http://www.containerstore.com/shop?productId=10035360&N=&Ntt=stacking+polystyrene+box">clear plastic polystyrene box</a> I purchased at the Container Store, although a small plastic leftover box will also work well.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Arduino Pro Mini Rainbow Kit Parts List</b></span><br />
<ol>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Arduino Compatible Pro Mini $2.00</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">10 element LED strip $2.70</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Solderless mini breadboard $0.80</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Battery holder $0.70</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Plastic Polystyrene Box $2.50</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Wire $.50</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">On-off switch $1.00</span></li>
</ol>
<span style="font-family: Arial, Helvetica, sans-serif;">
</span>
<span style="font-family: Arial, Helvetica, sans-serif;">Total: $11.40 (with 3 AA batteries) I have a detailed parts list <a href="https://docs.google.com/spreadsheets/d/1MREEQIvsVii4p6Q4gklcYhT_4hlK3q34k73b2Tp70W4/pubhtml">here</a></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I also suspect that if we purchase these in quantities of 20 or more the prices would come down a bit and we could get in under $10.<br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;">The biggest challenge is that each station must be equipped with a laptop with a <a href="http://www.ebay.com/itm/FTDI-BASIC-BREAKOUT-5V-USB-TO-TTL-6-PIN-MODULE-FOR-MWC-MULTIWII-LITE-SE-EM-01-/310938444698">FTDI basic breakout programmer</a> with the Arduino software pre-loaded. The Arduino Uno (which runs about $25) does not need the FTDI programmer, but would be 10x more expensive for each Rainbow kit. Getting these setup and configured is not easy.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I should also acknowledge that when we purchase items off of eBay or other sources, we are not using the <a href="http://store.arduino.cc/index.php?main_page=product_info&products_code=A000088">official mini</a> supplied by the fine people at Arduino. However at over $20 these devices would not fit into our under $10 budget. I still think that for other projects we should encourage people to use original Arduino hardware since the costs go to promote further open Arduino projects and quality educational materials.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Let me know your thoughts on this. Getting high-quality clear plastic boxes that kids could take home, throw in their back and show their friends is still something I am working on. I found one source <a href="http://www.transpack.co.uk/crystal-clear-presentation-boxes">here </a>that I might try but any suggestions you have would be great. I know other projects use simple baggies for their parts but these seem hard to use and show to others.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">There are also other options is to provide a "night light" mode that you could plug into a wall outlet and not have to use batteries. I have seen 5v USB "wall warts" for <a href="http://www.ebay.com/itm/1-50-USB-Power-Adapter-AC-CUBE-Wall-Charger-Plug-iPhone-3GS-4-4S-iPod-iTouch-USA-/281394533202">under $1</a>.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Although I know that this is a very small project to get kids started in Arduino, I think it might be the platform that other projects could be added. Creating a clock (12 LEDs required), putting sensors on to display temperature or adding accelerometers and magnetometers might be the next step for these kids. The key is that they could take them home and show their friends and that keeps them interested and motivate to continue their work.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I also want to thank Gerd Knops for helping me get started on using the LED strips. He is a brilliant engineer, a great friend and wonderful mentor to me. I hope that what I learn from Gerd I can use to get more kids interested in science, technology, engineering and math. We don't really have much time left to do this.</span><br />
<br />Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com3tag:blogger.com,1999:blog-13546516.post-9507844691578033142013-10-02T09:32:00.001-05:002013-10-02T09:32:18.658-05:00Agile Transformation in the Post NoSQL Era<span style="font-family: Arial, Helvetica, sans-serif;">Over the past four years we have seen the NoSQL movement grow from a small "Meetup" in the Bay Area to a technology that is touching all corners of the database world. Each year NoSQL software becomes more capable, lower cost and easier to use. Document stores, in particular, make the process of doing object-relational mapping unnecessary allowing anyone with god metadata (JSON and XML) to simply drag-and-drop their files into a centralized corporate data store. There are still a few challenges left. For example using statistical analysis of inserts and record counts to optimized indexes and putting good query languages (like <a href="http://jsoniq.com/">JSONiq</a>) on top of these data stores. But in large, these features are just polish on systems that are optimized to scale and be highly-available. The hard work seems like it has been done and we are now in the stage of refinement, not revolution.</span><div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">We documented the emergence of the NoSQL database patterns in our book, <a href="http://manning.com/mccreary">Making Sense of NoSQL</a>, which now available through Manning Publications. If you read this book you know that NoSQL systems have a diverse set of architectural patterns and different patterns apply to different problems. Selecting the right database architecture is a complex process of carefully understanding the subtleties of requirements and weighing the alternatives. Yet they do work well and once they are setup and configured they make data persistence a straightforward process.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">So whats next?</span><div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">Now that saving data has shifted from a project in its own to a smaller part of the application developer project we see the skills needed to build applications starting to shift. The need to model your data with ER modeling tools is getting less. The need to write complex joins with SQL is no longer needed. The next major skill set we would like to address is the the movement to agile transformation. How do you get you data out of your database and how do you transform it into the many formats that your application needs?</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">We think that the answer to this question is clear. Organizations need to be better at transforming the data in their database to other forms. This is the shift in skill sets from persistence centric to transformation centric. And it is not just the software developers that need to be able to transform data. Everyone on your team including non-programmers can play a role. They all need skills to quickly transform data from one form to another.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">We call this shift the movement to toward <b>Agile Transformation</b>. We hope to document how organizations are waking up to this new movement and understand the tools and processes they are adopting to empower everyone on their team to quickly transform data.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">This process of data extraction and transformation used to be a two-step process. SQL developers might create a series of tabular reports. These reports were then converted into the medium needed, HTML, XML, JSON, or even CSV files and other structures needed by other tools. Now the extract and transformation process can be done with a single step. Query results are no longer restricted to tabular formats.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif; font-size: large;"><b>Strategies for Agile Transformation</b></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">Over the next few months (or perhaps years), we hope to document many of the ways that organizations are attacking the agility challenges. Here are just a few strategies to get us started.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Single Source Canonical Data Models</b></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">If you are in the content management business you know that the concept of single-source publishing is central to your productivity. Using a single format to store content gets around the many-to-many transform problems that can drag down a teams productivity. We see the same principals also applying to web applications in general. Getting many data sources into a single format and then transforming this single format into many forms is the key to organizational productivity. We call these models "Canonical" since they are the standards that organization can build publish/subscribe web services around.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Flexible Query Languages</b></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">If you have every worked with tools like XQuery and JSONiq you know that they are the most flexible query languages around. These languages have benefited from years of work combining the best features of SQL, XSLT, XPath and dozens of other advanced query languages into a grammar that is designed to transform a variety of use cases.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Reusable Transformation Libraries</b></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">One of the first strategies that companies find is that many transformations are similar and can benefit from reusable code. Languages like SQL do offer a wide variety of non-portable stored procedures. Yet most of these languages limit your ability to build reusable transformation functions and modules. Modern languages need to be close enough to your data to understand how queries use indexes but abstract enough to be reused in new applications.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Using Great Tools: IDEs and Report Writers</b></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">SQL GUI Report Writers were one of the first tools that tool the complexity out of transforming tabular data. And we need more tools like these to make NoSQL reporting accessible to non-programmers. Some NoSQL products like HBASE already have SQL-like query tools. From our other blog posts you may know that we are big fans of the the oXygen IDE for managing JSON and XML data. oXygen makes the process of learning how to write XPath expressions easy for even the non-programmer. Even if their data is complex. These tools are complex in themselves and require hands-on training if non-programmers are going to get the most out of them.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Simplicity for Non-Programmers</b></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">One of the core foundations of agility is not have all your transformation be controlled by a small group of overworked developers. We learned that simple tools like GUI-based report writers or simple XPath templates can empower a non-programmer, with a bit of training, to build and maintain their own data transformations. Not needing to understand database joins is a big step in empowerment. Getting a good foundation library is another great step. Setting up small but easy to use templates is another good strategy. Building a search system to find the right library and templates also helps empower new staff and lowers the training burden on existing staff. In general, we feel that if a user "knows their data" that they should be given the tools to transform their data.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">So what is the best practices to build an organization that has agile transformation competency? We would love to know your ideas. Please send us email or tweet us at @dmccreary on Twitter.</span><br /></div>
</div>
Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-91058147707110693882013-07-01T08:56:00.000-05:002013-07-01T08:56:02.599-05:00Rounding error in Java when converting strings to doubles<span style="font-family: Arial, Helvetica, sans-serif;">I came across a rounding error when I was running an XQuery.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;">Here is the error:</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;">xquery version "1.0";</span><br />
<span style="font-family: Courier New, Courier, monospace;">number(3.1) + number(3.2)</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">which returned:</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;">6.300000000000001</span><br />
<br />
Not the expected value of 6.3. Note that the XQuery function number() returns double precision data.<br />
<br />
After a note from Eric Bruchez he suggested I cast the numbers to decimal:<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace;">xquery version "1.0";</span><br />
<span style="font-family: Courier New, Courier, monospace;">xs:decimal(3.1) + xs:decimal(3.2)</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;">and the problem seems to go away. He also showed that he could reproduce this error in other Java JVM languages like Scala.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;">Let me know if anyone else has seen this problem before and has any other suggestions for a fix.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span> <span style="font-family: Arial, Helvetica, sans-serif;">Thanks! - Dan</span>Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-6153617509003922482013-05-14T09:51:00.001-05:002013-05-14T09:51:32.527-05:00Analytical Reports for Technical BooksWe are wrapping up our work on our book "Makings Sense of NoSQL" and I have created a series of XQuery reports on our book that we have found useful.<br />
<br />
Our book is stored in XML DocBook format, which for those of you that have not used it, is ideal for technical books. DocBook contains elements for almost everything you need in a technical book including chapters, sections, paragraphs, figures, glossary terms, bibliographic references, and index terms. In short DocBook is is the perfect fit for most technical books and it can easily be extended. One key aspect about DocBook is that it is easy to transform into multiple formats such as HTML, PDF or ePub. There are many open source transforms available for DocBook. DocBook is at the heart of the movement into single-source publishing for technical publications.<br />
<br />
In additional to the standard DocBook transforms we also created a series of reports on the book and I thought they might be of interested to others. Here is a summary of some of these reports.<br />
<h2>
Chapter length report</h2>
When we started writing our book our goal was to produce a 310 page book with 12 chapters, each with approximately equal length. But logically we wanted our fist chapter to be a brief overview and we found that the chapter that described the core NoSQL patterns had more content.<br />
<br />
This is a horizontal bar chart that shows the length of each chapter. The goal is that all chapters be roughly the same size in length.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfb8tMXwYsXo86mUvLXJ1Rzd4EPune7MN_HxlomtEKpu6_VIKoh-TarWx9Hu2HRFbztB0lPT_eLAjwutRLWQwAjj9hI8tINAZ7WUCFPkzjgr4L_0c4XDtMyGQzdgQFDUiocSIBUg/s1600/chapter-length-report.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="138" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfb8tMXwYsXo86mUvLXJ1Rzd4EPune7MN_HxlomtEKpu6_VIKoh-TarWx9Hu2HRFbztB0lPT_eLAjwutRLWQwAjj9hI8tINAZ7WUCFPkzjgr4L_0c4XDtMyGQzdgQFDUiocSIBUg/s400/chapter-length-report.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<b>Chapter Length Report</b></div>
<div class="separator" style="clear: both; text-align: center;">
<b><br /></b></div>
You can see that one chapter in our book (chapter 4) is a bit longer then the rest of the chapters. We knew this up front after running this report and were prepared to defined our decisions with our editors.<br />
<h2>
The "Naughty words" report</h2>
We quickly found out that editors have some words that they don't want to see in a technical book. Words such as "just" or "very" should be used with great caution. There were also words that were not allowed "vs.", "e.g.", "etc.") according to the style guide. This report shows you how often you use these words in each chapter.<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2bn4vcQoMOgCch5HjNgE-FkmseUD3RuIv6RlFtWCfsfSggoQHiqUZuj0O_Em9SDUSJ97IQxHMn7a1jUlzfMR4xcKNjfj2ragDV91pb7UqUpa83_PVwM3FFModxUM1sm5PNVSoIg/s1600/naughty-words.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="281" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2bn4vcQoMOgCch5HjNgE-FkmseUD3RuIv6RlFtWCfsfSggoQHiqUZuj0O_Em9SDUSJ97IQxHMn7a1jUlzfMR4xcKNjfj2ragDV91pb7UqUpa83_PVwM3FFModxUM1sm5PNVSoIg/s640/naughty-words.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b>Report counting specific words in each chapter</b></div>
<br />
<h2>
Book metrics report</h2>
This is just a raw counts of elements such as book parts, chapters, sections, figures and tables etc.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5MxsXlhegrQkjlVFr9Ic0IHsa89FyQISbnhifvdhFYv9LPMVlAc_Ffj4MWhBJDOSOK_BJC3xstBU3gCj9l8alfvBsSK3kvrsjbiAFkq3G5Zm2iJjebQoVsCOyReAixBw08BJgGA/s1600/book-metrics.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5MxsXlhegrQkjlVFr9Ic0IHsa89FyQISbnhifvdhFYv9LPMVlAc_Ffj4MWhBJDOSOK_BJC3xstBU3gCj9l8alfvBsSK3kvrsjbiAFkq3G5Zm2iJjebQoVsCOyReAixBw08BJgGA/s320/book-metrics.png" width="148" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b>Book Metrics Report</b></div>
<br />
<h2>
Chapter metrics report</h2>
For each chapter I created a detailed report of the content. As you are writing each chapter, other people can view your progress by running this report. I tend to put in outlines first, then figures and then the actual text of each chapter.<br />
<br />
<h2>
List of figures and tables by chapter</h2>
These reports shows a listing of figures and tables sorted by chapter and location in each chapter. It also shows the caption for each figure and a thumbnail of the image. There are versions that show the type of figure (line-art vs. bitmapped) the sources of each figure.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi5ZaQvnqex1A1UoDIYqrkFKapCCi-xYM8AjntZ5f8Lq9PM6RVFUdSeIZ4WWylJ1LkOln3u4I8dw7xW6CWbmSwRWBYrhyZPOisZd-p2wYEzNlWQ_gdP3sDFaFoxpneU99BJQj_Wg/s1600/list-of-figures.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="344" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgi5ZaQvnqex1A1UoDIYqrkFKapCCi-xYM8AjntZ5f8Lq9PM6RVFUdSeIZ4WWylJ1LkOln3u4I8dw7xW6CWbmSwRWBYrhyZPOisZd-p2wYEzNlWQ_gdP3sDFaFoxpneU99BJQj_Wg/s640/list-of-figures.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b>List of Figures Report</b></div>
<br />
<h2>
Figure and table captions reports</h2>
Editors want to make sure a book is "browsable" which means that every other page has an interesting figure that people will see when they flip through the book. Each of these reports can list the length of the figures and table captions sorted by the length of the caption. The captions that are too short will need further work.<br />
<br />
<h2>
Paragraph and sentence length histogram reports</h2>
There are guidelines that should be used when writing technical books on sentence and paragraph length. This report shows the distribution of paragraph lengths in each chapter. You should work with your editor to find reasonable guidelines for your audience.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQ-QN684AUSMVsuJNISk4czSU6-pDSIMv8aJW8Ncqm9UqUyy6O6TY_P8upHuR7KDO4d8_BbaDFrqfSVcKrIs-cSN7G_fSquz9_9O0LPMsD3wSdpCdAY0Ynw_yUO10teA4rzYA7sA/s1600/paragraph-histogram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQ-QN684AUSMVsuJNISk4czSU6-pDSIMv8aJW8Ncqm9UqUyy6O6TY_P8upHuR7KDO4d8_BbaDFrqfSVcKrIs-cSN7G_fSquz9_9O0LPMsD3wSdpCdAY0Ynw_yUO10teA4rzYA7sA/s640/paragraph-histogram.png" width="498" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b>Paragraph Size Distribution Report</b></div>
<br />
<h2>
Table of contents reports</h2>
For reviewing the book structure it is always nice to have reports that list the book's structure by chapter, section, sub-section and sub-sub-section. These reports come with several parameters that limit the depth of the table of content and can also be modified to create mind maps using open source mind mapping tools.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLdLp8nqM5R8YqzbiGQKWxdFwcav5B_nONnbWaO_YuUSjJcBgGBRxWGUlGTWr6oJyVUoETJAMdFPxj2IBqzAMIhp5EWlRwMH_IOZOz1CUiwsw6KfAl-uOSlA3qkBmEvtOjLfuegA/s1600/full-toc.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="204" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLdLp8nqM5R8YqzbiGQKWxdFwcav5B_nONnbWaO_YuUSjJcBgGBRxWGUlGTWr6oJyVUoETJAMdFPxj2IBqzAMIhp5EWlRwMH_IOZOz1CUiwsw6KfAl-uOSlA3qkBmEvtOjLfuegA/s320/full-toc.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b>Table of contents showing Parts, Chapters, Section 1 and Section 2</b></div>
<br />
Once I wrote the basic structure of the table-of-contents report it was easy to create other output formats for different functions. Here is an example of a GraphViz output.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyh8OmbyRGlP5WgCVt2hRdvhagXY2e_FQnKxrYjqu-iGZ2Bn9uMQ0J_FktjIV9bs2esoFDssIm7aZOvFtDUAzr4ZmuU_4lQDF7kVBJEL_iuA6fa1whxzp6hn6SGJgIK-9sdvXSlQ/s1600/mind-map.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="178" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyh8OmbyRGlP5WgCVt2hRdvhagXY2e_FQnKxrYjqu-iGZ2Bn9uMQ0J_FktjIV9bs2esoFDssIm7aZOvFtDUAzr4ZmuU_4lQDF7kVBJEL_iuA6fa1whxzp6hn6SGJgIK-9sdvXSlQ/s320/mind-map.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b>Book Outline using GraphViz format.</b></div>
<br />
This example used the <a href="http://www.martin-loetzsch.de/DOTML">DOTML </a>markup format and an external transform using the <a href="https://github.com/KitWallace/graphviz">Chris Wallace graphviz XQuery module</a>.<br />
<br />
You can also convert the table of contents into an Mindmap file and open the file in FreeMind or XMind.<br />
<br />
Here is an example of the book rendered in XMind. <a href="http://danmccreary.com/books/nosql/making-sense-of-nosql-mindmap.png">NoSQL Book MindMap</a><br />
<h2>
Glossary of terms reports</h2>
Our book puts a focus on the terminology used in the NoSQL movement. We try to create precise definitions of all the terms we use and discuss the variations in definitions in different NoSQL communities. I use these reports to list each time a term is first introduced and make sure that we have a formal definition in the Glossary Appendix at the end of the book.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBryEsq-Z8OJGKeA79d-ojAoZcgKaHArVe6xprrpMot6Qw9EB6PjDyotqzGaBxbxDbGW6aVFHkxa8OJWlz_JCDsm_VyzpIWj0N3qsYF2amYxhFqO-CrRe5MsXFiFIQgtPFDXjS6w/s1600/glossterm.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="232" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBryEsq-Z8OJGKeA79d-ojAoZcgKaHArVe6xprrpMot6Qw9EB6PjDyotqzGaBxbxDbGW6aVFHkxa8OJWlz_JCDsm_VyzpIWj0N3qsYF2amYxhFqO-CrRe5MsXFiFIQgtPFDXjS6w/s640/glossterm.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b>Glossary Term Report Showing Glossterm IDs and Definition Status</b></div>
<br />
There are also other miscellaneous reports listing the introductory chapter quotes, lists of comments by reviewer (extracted from PDF using Apache Tika), and various tools to help us gauge the completeness of each chapter.<br />
<br />
<h2>
Report Strategy</h2>
We created a central XQuery book module that had all the common functions such as <span style="font-family: Courier New, Courier, monospace;"><b>$book:chapters</b></span>, or that returned a sequence of all book chapters or <span style="font-family: Courier New, Courier, monospace;"><b>book:word-count($node)</b></span> that returned a word count of a node. I used the eXist-db database to store and execute the transforms. After we created the module the templates could be quickly customized for each report. We also used the oXygen XML IDE extensively and we want to thank George Bina for his support of our project.<br />
<br />
I think that DocBook, oXygen, XQuery and eXist are ideal tools for managing the book creation process.Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com1tag:blogger.com,1999:blog-13546516.post-80253352075245954192013-05-06T09:19:00.002-05:002013-05-06T09:19:24.450-05:00Cognitive Bias in NoSQL System Selection<br />
After attending the Saturn 2013 Conference I was exposed to the use of "Cognitive Bias" in software architecture.<br />
<br />
Here are some examples of cognitive bias I have seen as applied to the world of NoSQL database selection.<br />
<br />
<b>Anchoring bia</b>s - the tendency to produce an estimate near a cue amount - <i>"Our managers were expecting an RDBMS solution so that’s what we gave them."</i><br />
<br />
<b>Availability heuristic</b> - the tendency to estimate that what is easily remembered is more likely than that which is not. -<i> "I hear that NoSQL does not support ACID." or "I hear that XML is verbose?"</i><br />
<br />
<b>Bandwagon effect </b>- the tendency to do or believe what others do or believe - <i>"Everyone else at this company and in our local area uses RDBMSs."</i><br />
<br />
<b>Confirmation bias</b> - the tendency to seek out only that information that supports one's preconceptions – <i>"We only read posts from the Oracle|Microsoft|IBM groups."</i><br />
<br />
<b>Framing effect</b> - the tendency to react to how information is framed, beyond its factual content <i>"We know of some NoSQL projects that failed."</i><br />
<br />
<b>Gambler's fallacy</b> (aka sunk cost bias) the failure to reset one's expectations based on one's current situation –<i> "We already paid for our Oracle|Microsoft|IBM license so why spend more money?"</i><br />
<br />
<b>Hindsight bias -</b> the tendency to assess one's previous decisions as more efficacious than they were –<i> "Our last five systems worked on RDBMS solutions".</i><br />
<br />
<b>Halo effect</b> - the tendency to attribute unverified capabilities in a person based on an observed capability. – <i>"Oracle|Microsoft|IBM sells billions of dollars of licenses each year, how could so many people be wrong". </i><br />
<br />
<b>Representativeness heuristic</b> - the tendency to judge something as belonging to a class based on a few salient characteristics -<i> "Our accounting systems work on RDBMS so why not our product search?"</i><br />
<div>
<br /></div>
<div>
Thanks to everyone at CMU/SEI and the SATURN Conference for the exposure to these ideas.</div>
Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-41874038977303573732013-05-01T17:30:00.001-05:002013-05-03T04:38:48.888-05:00How to pass oXygen XML IDE Editor variables into an XQuery script.I am a big fan of the oXygen XML Integrated Development Environment. It has many powerful features and I learn about new ones the more I use it. I recently started a project that required me to run XQuery transform on hard-drive files using Saxon HE. Since I normally run XQuery from within the eXist-db this was a new challenge for me.<br />
<br />
I have many XML files and many different XQueries and I wanted to run them directly from test cases in an oXygen project. But unlike XSLT, XQuery does not have a concept of "standard input". You need to specify a path to the document within the XQuery file itself. So how to pass the file name from the project directly to Saxon? Luckily, the guys from Syncro Soft (the authors of oXygen) thought about this. You can use oXygen "<a href="http://www.oxygenxml.com/doc/ug-editor/#topics/editor-variables.html">Editor Variables</a>" to be parameters to your XQuery.<br />
<br />
Here is a sample program that reads two external variables, one for the project director and one for the current file name (with extension).<br />
<h3>
XQuery Program</h3>
<pre>xquery version "1.0";
<span style="color: #274e13;">
</span></pre>
<pre>(: Example of how to access external "Editor Variable" variables passed</pre>
<pre><span style="color: #274e13;"> </span><span style="color: #274e13;">by </span><span style="color: #274e13;">oXygen. This one example gets us to a full file path within a</span></pre>
<pre><span style="color: #274e13;"> </span><span style="color: #274e13;">project. </span><span style="color: #274e13;">See:</span></pre>
<pre><span style="color: #274e13;">
</span></pre>
<pre><span style="color: #274e13;">http://www.oxygenxml.com/doc/ug-editor/topics/editor-variables.html </span></pre>
<pre><span style="color: #274e13;">
</span></pre>
<pre><span style="color: #274e13;"> for other editor variables.
:)
</span><b><span style="color: #274e13;">(: oXygen project directory :)</span>
declare variable $pd as xs:string <span style="color: red;">external</span>;</b>
<span style="color: #274e13;">(: file name with extension :)</span>
<b>declare variable $cfne as xs:string <span style="color: red;">external</span>;</b>
<span style="color: #274e13;">(: note the triple forward slashes before the file. :)</span>
let $path := concat('<span style="color: red;">file:///</span>', $pd, '\', $cfne)
return
<results><results>
<project-dir>{$pd}</project-dir>
<file-name>{$cfne}</file-name>
<file-path>{$path}</file-path>
<doc-available>{<span style="color: #274e13;">doc-available($path)</span>}</doc-available>
</results>
</results></pre>
<br />
Note that we put the project and file together to get the path to the file and use the doc-available() function to verify that the file is there.<br />
<br />
Next, we set up a transformation scenario and put the file above in the transform URL. We have to add the parameters to the transform. oXygen parses your XQuery for all external variables and puts them in each line for you. For each variable put in the same variable with the ${pd} notation where the dollar sign is outside the curly braces (opposite of XQuery).<br />
<br />
Here is a screen image of these parameters and their values.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBV0j7CgGbyMYUSTCNY8ELjgki_2cRq8ASP7gTxgjyvWPQ-3jQ_NEheXQ-wRJsU_RS4aYeWDQCOPeq_VR48zzuZlGNSuUIhXX_x1-NbN6XbWXydlywhL6Tlj4j0rjN2sHku83Wtg/s1600/xquery-editor-variables.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBV0j7CgGbyMYUSTCNY8ELjgki_2cRq8ASP7gTxgjyvWPQ-3jQ_NEheXQ-wRJsU_RS4aYeWDQCOPeq_VR48zzuZlGNSuUIhXX_x1-NbN6XbWXydlywhL6Tlj4j0rjN2sHku83Wtg/s640/xquery-editor-variables.png" width="640" /></a></div>
<br />
<h3>
Results</h3>
When we run this transform on any file on Windows we get the following:<br />
<br />
<results><br />
<project-dir>D:\ws\project\subdir</project-dir><br />
<file-name>my-input-file.xml</file-name><br />
<file-path>file:///D:\ws\project\subdir\my-input-file.xml</file-path><br />
<doc-available>true</doc-available><br />
</results><br />
<br />
On Mac and UNIX systems we would not use the file:/// prefix and the separators will be forward slashes.Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-77316006061911942962012-08-17T12:30:00.000-05:002012-08-17T12:30:22.308-05:00NoSQL book now available in MEAPThe book that Ann Kelly (my wife) and I have been working on for the last year is now available on the Manning Early Access Program (MEAP). Here is the link:<br />
<br />
<a href="http://">http://manning.com/mccreary</a><br />
<br />
The final title of the book will be:<br />
<br />
<b>Making Sense of NoSQL: A guide for managers and the rest of us</b><br />
<br />
You can download chapter 1 for free but to get the other three chapters you will need to be part of the MEAP program which is priced at $23.99 for the Ebook only and $29.99 if you would also like a printed copy sent to you when the book is done.<br />
<br />
We are looking for reviewers of the book. We are especially interested in finding anyone that has an interest in how to explain NoSQL concepts to non-programmers. This includes Managers, Project Managers, Business Analysts and anyone else that is interested in using NoSQL.<br />
<br />
I believe this this is one of the first NoSQL books that is targeting managers and other non-programmers. In order to write to a general audience we are trying to use extensive tables and annotated graphics to make a highly readable book that give people an overall understanding of the NoSQL movement. Much of these tools have been developed by Ann and myself over the last several years trying to explain NoSQL concepts as part of our consulting business.<br />
<br />
If you know of good NoSQL metaphors or even NoSQL jokes that we can include in the book please let us know! We will be very involved in the Manning Author feedback program.Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-84562615325994417362011-09-30T08:35:00.004-05:002011-09-30T23:13:20.149-05:00AnyChart Case StudyI just released a case study (click on the blog post title or click <a href="http://www.anychart.com/blog/2011/09/22/building-a-large-chart-ecosystem-with-anychart-and-native-xml-databases/">here</a>) using the commercial software charting package called "AnyChart". After using over a dozen charting solutions including Google Charts and FreeCharts I really feel that AnyChart has the best solution that really scales up. Not only does AnyChart have a large number of charts but it is also very reasonably priced at around $500USD per server. Because I store all the chart specification and implementations in XML and I use <a href="http://exist-db.org/">eXist </a>to manage the charts it is easy to manage the full lifecycle from requirements to testing all within a single Lucene-search powered environment.<br />
<br />
The latest release of AnyChart also generates SVG and HTML5 compliant charts that run on the iPhone and iPad. This really shows how "declarative" systems are much more portable than any flash-based implementations.
<br />
<br />
If you are looking for an inexpensive but powerful tool for building charts and dashboards I would defiantly give them a try.
<br />
<br />
You can get samples at their web site at <a href="http://anychart.com/">anychart.com</a>Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-38632982887561193462011-09-24T08:33:00.007-05:002011-09-24T13:10:48.238-05:00The "Pathification" of NoSQLIn my consulting work I try to focus on the empowerment of the non-programmer to build and maintain their own web applications. This is not an easy task. There are many barriers, some political and some technical. But when a new Business Analysts or Project Manager can create their own metadata management tools and begin to tell others their story I have great job satisfaction.
<br/><br/>
My current best-practice is to teach non-programmers how to write XPath expressions into their data sets. I tell them that if they are committed to "knowing their data" that I can teach them how to write web-applications in a week or so. The trick is to give them a "template" CRUDS application (Create, Read, Update, Delete, Search) that they can just plug in their own XPath expressions to get the application to work. They don't need to know the thousands of features of XQuery and native XML databases, they just need to know their data, how to write some simple XPath expressions and where to put these expressions into the templates.
<br/><br/>
I call this process "Pathification", a word that is similar to "Simplification" and "Standardization" but it has a focus on only teaching a very small subset of the many possible tools and a focus on cross database standards like XPath. Once they feel confident they can create simple CRUDS applications in a few hours they are then highly motivated to continue learning XQuery, but on their own pace and balanced with their other job responsibilities like data governance and semantics.
<br/><br/>
Pathification is something that can happen because native XML database have many enabling technologies. They have WebDAV drivers that makes you database look like folders on your desktop. We have tools that automatically convert XML into HTML and XForms. We have subversion libraries that allow your form "save" to go directly into a version control system. And these tools continually get better and make it easier to empower novice users.
<br/><br/>
The NoSQL movement is a huge and diverse community. And many people have focused on areas such as scalability and the integration of different types of data into a single XML development environment. These are all great efforts and really move us in the right direction. But I think for us to make NoSQL into a true multi-vendor, multi-product development platform is to create more tools that empower the non-programmer. For people in the native XML world we have a great start. Standards like <a href="http://expath.org/">EXPath</a>, new application packaging tools and XRX <a href="http://atomic.exist-db.org/blogs/eXist/?id=urn:uuid:09bcc0de-a868-48e9-9fef-ceeb032028c3">application repositories</a> will continue to make it easier for non-programmers to reuse and share their applications.
<br/><br/>
What I would like to see (but I am not sure is possible) is both XML and JSON-centric system to start to come together and share their ideas on how to make this happen. Can JSON data-stores like MongoDB and CouchDB be "Pathified"? Can we create shared applications that work with both types of back-end systems?
<br/><br/>
If we do this right, I think the world will be a much better place. People will have more choices for managing both structured and semi-structured data and non-programmers will be able to take a spreadsheet or new data feed in a standard format and build a new web application that manages this data in just a few hours without having to depend on other programmers. I think of this as a system as easy to use as Microsoft Access(TM) but without the chaos: All data can be stored on centralized servers with REST service-oriented interfaces for every data set.
<br/><br/>
Please let me know your thoughts on these ideas.Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com3tag:blogger.com,1999:blog-13546516.post-60357574870393699222010-09-24T08:05:00.003-05:002010-09-24T08:25:28.590-05:00McCreary's Law<p>I am just returning from a three-day conference on Enterprise Information Management. I was very pleasantly surprised to meet many other very smart and warm independent consultants that do metadata and EIM consulting. More posts about what they taught me will be coming.</p>
<p>But these conferences for me to constantly answer the same question over and over again. What do I do an why don't I use traditional RDBMS systems to store metadata? What I need is some strong statement that explains the core principal of the need for agility in metadata management. So in that context, I would like to suggest a law of agile software development. To give it a label, I will just use <b>McCreary's Law for now.</b></p>
<p>McCreary's Law states the following:</p>
<p><i>
The agility of any software project in inversely proportional to the square of the number of data transformations in the developer stack.</i></p>
<p>
The number of data transformation in the traditional three tier-architecture (Web to Objects to RDBMS to Objects and back to web) is four.</p>
<p>The number of data transformation in the traditional three-tier plus XML web services is six, since web services are usually created by one transform from objects to XML and one from XML to objects.</p>
<p>The number of data transformations in XRX is approximately zero, since we could make an argument that XML to HTML is a 1/2 translation since the HTML and XML tags are different. Of course if you are not changing the order all XML can be styled with CSS which is not really a transformation. This gets around the problem of infinite developer agility if the denominator is zero.</p>
<p>This formula is not perfect and I am sure their will be modifications to it like adding a constant for training and IT resistance to change, but I hope people can use it to predict or explain their current development agility.</p>
<p>Thanks to everyone I met at the EIM conference that listened to my rants and have not discarded me as a complete metadata radical.</p>Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com2tag:blogger.com,1999:blog-13546516.post-78026352738405666532010-09-12T07:27:00.007-05:002010-09-12T07:50:33.162-05:00XLucene<p>For the past two years I have been quietly observing the growth of something called <span style="font-style: italic;">Structured Search</span>. This is the use of document structure to aid in search and retrieval of documents or sub-documents in large document collections.
</p>
<p>I have also seen a common solution to this problem. This is to use a combination of native XML database and the Lucene keyword indexing problem. But people doing this are doing far more than just simple keyword search. So calling this new approach simply “Lucene” keyword search does not really describe what is going on. So I am proposing an new meme: XLucene.
</p>
<p>I define XLucene as using a combination of document structure and keyword searches together to create very precise search ranking or search hit scores. The key is that the design must be able to store each branch in the document tree as a separate subdocument.
</p>
<p>The key factor in widespread adoption is that almost anyone that can identify the tag names in their document structures will soon be able to create their own XLucene search and retrieval systems.
</p>
<p>What first got me interested in “Structure Retrieval” was the book “Introduction to Information Retrieval” by Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze. In this book they present the following table:
<table style="border: 2px solid black;" border="1" cellpadding="3">
<tbody>
<tr><td align="LEFT">
</td>
<td align="LEFT"><span style="font-weight:bold;">RDB search</span></td>
<td align="LEFT"><span style="font-weight:bold;">unstructured retrieval</span></td>
<td align="LEFT"><span style="font-weight:bold;">structured retrieval</span></td>
</tr>
<tr><td align="LEFT">objects</td>
<td align="LEFT">records</td>
<td align="LEFT">unstructured documents</td>
<td align="LEFT">trees with text at leaves</td>
</tr>
<tr><td align="LEFT">model</td>
<td align="LEFT">relational model</td>
<td align="LEFT">vector space & others</td>
<td align="LEFT">?</td>
</tr>
<tr><td align="LEFT">main data structure</td>
<td align="LEFT">table</td>
<td align="LEFT">inverted index</td>
<td align="LEFT">?</td>
</tr>
<tr><td align="LEFT">queries</td>
<td align="LEFT">SQL</td>
<td align="LEFT"><a name="12114"></a>free text queries</td>
<td align="LEFT">?</td>
</tr>
</tbody></table>
The following quote follows this table:
<span style="font-style: italic;">There is no consensus yet as to which methods work best for structured retrieval although many researchers believe that XQuery will become the standard for structured queries. </span>
</p>
<p>I think that the jury has returned their verdict. XQuery wins! Here is that final table with the following properties:
</p>
<p>
<span style="font-weight: bold;">Objects:</span> Trees with text at leaves<br/>
<span style="font-weight: bold;">Model:</span> Hierarchical documents<br/>
<span style="font-weight: bold;">Main-data-structure:</span> trees and inverted indexes tied to node-ids<br/>
<span style="font-weight: bold;">Queries:</span> W3C XQuery with fulltext extensions<br/>
</p>
<p>The design of using variable-length node-ids as the document identifiers was done by Wolfgang Meier of the eXist-db.org project. There are many people now using this design today. I will be following up this posting with some case studies of how people are using this in several areas, both in the government sector, library metadata and document search and retrieval.
</p>
<p>Since there are so many people using this same problem-solution pair, that like all good patterns, it deserves its own name. XLucene (with no embedded dash) can be picked up by search engines and we can all start to share design experience and solutions. This is why the Open Source model really is superior to closed systems. Knowledge just travels faster once the memes are created.
</p>
<p>I will also be presenting more details on the general topic of <a href="http://eim2010.wilshireconferences.com/sessionPop.cfm?confid=48&proposalid=3357">Structured Search</a> and XLucene at the Enterprise Information Management Conference in Toronto next Tuesday, Sept 21st.</p>Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com1tag:blogger.com,1999:blog-13546516.post-7670654828442254632010-08-30T20:04:00.007-05:002010-08-30T20:22:24.173-05:00What Makes a Good Solution Architect?<p>A recruiter friend of mine was looking for a senior "Solution Architect" and needed some help setting up a screening interview. Here is what I told him.</p>
<p>I look for solid experience in a diversity of solution. There are just too many people that think that HTML/Objects and RDBMS systems are the only tools in your architect's toolkit.
</p>
<p>
Here is what I suggested the interview questions would be like:</p>
<p>
Consider the following application architectures:</p>
<ol>
<li>Ruby on Rails on a relational database</li>
<li> XRX on a native XML database</li>
<li>A Java Client on J2EE app server using JMS</li>
<li>A JQuery client on an OLAP database</li>
</ol>
<p>
A senior "Solution Architect" should be able to compare the pros and cons of each system and describe how they have used each of these architectures to build solutions and give organizations a competitive advantage. I also drill down into their understanding of search and retrieval and natural language processing.</p>
<p>
As a bonus question, I would ask how each architecture would impact an organization's business strategy and to what extent business units could be empowered to build their own applications with minimal training and little or no need for IT involvement.</p>
<p>
Most good solution architects should have experience with 3 out of 4 solution architectures but very few usually have a strong MBA-type background to understand how solution architectures impact business strategy. And if you find anyone that has built systems with all for and has an MBA you have a serious senior solution architect. Hire them!</p>Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-31475771985578511532010-07-02T10:15:00.006-05:002010-07-02T10:50:01.745-05:00Impressions of SemTech 2010Two weeks ago I attended my fifth Semantic Technology conference in San Francisco.<span style=""> </span>It was a great conference!<span style=""> </span>This is my fifth time attending the conference and I plan to attend in the future!<span style=""> </span>My biggest dilemma was which session to attend. At times there were as many as eight concurrent sessions going on.</p> <p class="MsoNormal">There were a few big trends that I spotted.</p> <p class="MsoNormal">The use of RDFa to annotate web pages using semantically precise elements was clearly a big trend.<span style=""> </span><a href="http://twitter.com/jaymyers">Jay Myer</a> of <a href="http://www.bestbuy.com/">BestBuy </a>described how the BestBuy sales went up 30% when they added RDFa tags to their product pages.<span style=""> </span>Although many search engines are not transparent about their use of RDFa tags in page rankings, Jay’s results should make it clear that this strategy works.</p> <p class="MsoNormal">Jay told a great story about how hard it was to find a fridge that met his specific criteria: black with a specific size etc. using a keyword-based search engine.<span style=""> </span>The semantic web will change all of this!</p><p class="MsoNormal">The big factor here is <a href="http://www.heppnetz.de/">Martin Hepps</a> <a href="http://www.heppnetz.de/projects/goodrelations/">GoodRelations </a>ontology for products and services. This can really bring the Semantic Web to the masses. Finally a way to code the hours of operation for your store so that you can use <a href="http://siri.com/">Siri </a>to ask "What Sushi Restrauants are Open at 10pm Near Here"
</p> <p class="MsoNormal">There were also a lot of sessions on <a href="http://en.wikipedia.org/wiki/Linked_Data">LinkedData </a>and specifically on LinkedData in the government area.<span style=""> </span>Jim Heldler and his students at RPI are scraping every data set on data.gov and converting them all to RDF and storing them in a huge triple store.</p> <p class="MsoNormal">Jim also provided one of the best sound-bites from the conference:</p> <p class="MsoNormal">“Get AWAY <span style=""> </span>from the Table”</p><p class="MsoNormal">This goes far beyond the <a href="http://en.wikipedia.org/wiki/NoSQL">NOSQL </a>movement that we are seeing. It gets to the core of the problems with innovation in many organizations.
</p> <p class="MsoNormal">Jim is a consultant to the US data.gov transparency process and a key advocate of open linked data standards.<span style=""> </span>It is interesting to see a friendly rivalry between Jim and Tim Berners-Lee who is a consultant for the UK data transparency movement.<span style=""> </span>Both are using RDF to convert data but each is using slightly different strategic approaches.</p> <p class="MsoNormal">He was referring to the fact that many organizations over-use the relational model and they need to understand that there are many alternatives, especially when doing mashups of data sets from many sources.</p> <p class="MsoNormal">There were also many presentations on natural language processing and finding the true “meaning” of words within unstructured and semi-structured data sets. </p> <p class="MsoNormal">I always attend <a href="http://www.bosatsu.net/">Brian Sletton</a>’s session on REST.<span style=""> </span>Many people do not understand the relationship between REST, URI’s and the semantic web stack.<span style=""> </span><a href="http://history.state.gov/">Joe Wicentowski’s work</a> at the US Department of State on the URL rewriting frameworks within eXist has really reinforced how this can be done easily in a single XQuery module.</p> <p class="MsoNormal">I presented a 3.5 hour tutorial session on Entity Extraction.<span style=""> </span>Since it was scheduled for the last session of the last day of the conference I though the attendance would be very low.<span style=""> </span>But the room was packed and most of the people stayed till the very end.</p> <p class="MsoNormal">It was wonderful to finally meet Marie Wallace and DJ <span class="gi">McCloskey </span>from the <a href="http://www-01.ibm.com/software/globalization/topics/languageware/index.jsp">LanguageWare </a>team at IBM.<span style=""> </span>Their support of the Apache UIMA standards will be a great step forward to the creation of interoperable language analysis piplelines.<span style=""> </span>Marie and DJ introduced me to the Millennium restaurant just a few blocks from the hotel. <span style=""> </span>My wife Ann and I went there over the weekend and we now own one of their cookbooks. </p> <p class="MsoNormal">By the way Google is now also support recipes in their <a href="http://www.google.com/support/webmasters/bin/answer.py?answer=99170">rich snippets</a> in their search results so you will soon be able to find "only recipes that take under 30 minutes" in a Google search engine.</p> <p class="MsoNormal">There were also many session on the need for good taxonomies and ontologies in the enterprise.<span style=""> </span>The use of <a href="http://www.w3.org/2004/02/skos/">SKOS </a>for controlled vocabularies was a very hot topic and there were dozens of presentation on how OWL is being used to capture and exchange business rules.</p> <p class="MsoNormal">It was also really great to finally meet <a href="http://www.jenitennison.com/">Jeni Tennison</a> after reading her books and following her Tweets for a long time.<span style=""> </span>She is deeply involved in the federal data transparency projects in the UK.<span style=""> </span>Here use of RDF is breaking new best practices for the entire community.</p> <p class="MsoNormal">It was also great to catch up with Mark Birbeck, one of the chief architects of the <a href="http://code.google.com/p/ubiquity-xforms/">Ubiquity XForms libraries</a>.<span style=""> </span>I am looking forward to trying out some of the new tools based on his backplane JavaScript libraries.</p> <p class="MsoNormal">The people from Facebook also gave a presentation on how they are adding the ability for people to add a few lines to each web site to allow users to add a “like” button on a web site using a variation of RDF.<span style=""> </span>Many people were a little disappointed to hear that they will not be using namespaces in their interfaces.<span style=""> </span>Facebook felt that host HTML coders could only handle a single namespace.<span style=""> </span>Many people agreed with their findings and thought that until 90% of HTML coders knew what namespaces were that organizations like Facebook were stuck just adding new data to HTML meta elements.</p> <p class="MsoNormal">One of the most interesting discussions I had was with the people from <a href="http://www.cray.com/">Cray Computer</a>.<span style=""> </span>Apparently a large unnamed government agency has given Cray Research a very large contract to build customized ASIC (FPGA) chips to do graph analysis.<span style=""> </span>Their ThreadStorm <a href="http://www.cray.com/products/xmt/">XMT </a>architecture has 128 register sets and allows the CPUs to get continuous feeds of graph queries without waiting for memory.<span style=""> </span>Their claim is that the federal agency has told them they are getting a 100X improvement in complex graph queries.<span style=""> </span>The challenge is that the API is currently a low-level C interface and they have not yet put a SPARQL complier in front of an XML.<span style=""> </span>So on good SPARQL benchmarks are yet available to compare the results of this hardware with a typical triple store.<span style=""> </span>But even if it is fast, the price would be in the six digits for one of the XMP systems.<span style=""> </span>So only very large organizations and government agencies might use this unless it was provided as a service.</p> <p class="MsoNormal">One of the other things I found was how many people are using OWL and the OWL reasonerers like <a href="http://clarkparsia.com/pellet">Pellet </a>as replacements for traditional rules engines from companies like FairIsaac.<span style=""> </span>There are two reasons for this.<span style=""> </span>First is that many rules engine companies only talk about their engine performance but not on the need for precise semantics in the data elements.<span style=""> </span>With OWL and the methods behind the semantic web stack we attempt to put semantics higher in the requirements of a system a use pre-built and pretested ontologies.</p> <p class="MsoNormal">The second big reason that people are use OWL is that the rules are created using open standards.<span style=""> </span>That means you are not locked into any one vendors rule format.<span style=""> </span>The new W3C standards for rule interchange (RIF) will also start to break down many of the barriers to the creation and exchange of large industry vertical rule sets to that each organization does not have to start from scratch with a rule base.<span style=""> </span>This will be big for industries like insurance where claims processing ontologies are just being created.<span style=""> </span>Thanks to <a href="http://clarkparsia.com/weblog/2010/05/26/another-reason-semantic-web-kicks-ass/">Kendal Clark</a> for good information on this topic.</p> <p class="MsoNormal">I also had a great time talking to many people in the publishing area.<span style=""> </span>Seth Maislin gave a great presentation on Taxonomies.<span style=""> </span>Seth and <a href="http://thetaxonomyblog.wordpress.com/">Marlene Rockmore</a> joined me for a lunch at nearby Indian restaurant.<span style=""> </span>Marlene is an expert on Taxonomy development.<span style=""> </span>She was working on a book with O’Relly on taxonomies that is on hold for now but we hope to see one in the future from her.</p> <p class="MsoNormal">It was also nice to see how many federal agencies are now starting to use SPARQL in the intelligence community.<span style=""> </span>I saw a good presentation by <span class="entry-content">Dennis Wisnosky with the US DOD talk about how they plan to use NIEM and semantic technologies to cut their integration costs.<span style=""> </span>Yeah NIEM!<span style=""> </span>Most people don’t understand that the NEIM is based on the RDF model, it just uses XML syntax to store the relationships.<span style=""> </span>See my web-cast on the SemanticUniverse.com web site for more details. <span style=""> </span>Hopefully his slides will be available in the future.<span style=""> </span>Many people were taking photos of the slides since the DOD is not great at getting their slides out.<span style=""> </span>Wonder why?<o:p></o:p></span></p> <p class="MsoNormal"><span class="entry-content">I also had lunch with several people from within the intelligence community that had great discussions about the pros and cost of pulling assertions out of NLP-analyzed annotated XML documents and the challenges relating to this.<span style=""> </span>Keeping the links back to the original XML documents to is critical to verify the results of an assertion in context.<span style=""> </span>Traceability, linage, provenance and time-domain representations in RDF that don’t cause a 10X growth is the number of triples is a difficult problem.<o:p></o:p></span></p> <p class="MsoNormal"><span class="entry-content"><span style=""> </span>If there was some way to relate RDF assertion to a source document XML Node ID (the fourth column question) is a very difficult problem and one that needs close research with native XML and RDF systems.<span style=""> </span>One of the things I learned about the <a href="http://www.exist-db.org/lucene.html">eXist-Lucene</a> integration done by Wolfgang Meier is that keeping the node-id as the document ID in Lucene allows non-programmers the ability to configure customized search rank rules past on the context of the keyword in a document.<span style=""> </span>This is where highly customizable <a href="http://www.syntactica.com/solutions/structured-search.xq">structured search</a> rocks.<span style=""> </span>I think this innovation needs to be added to RDF triples projects.<span style=""> </span>Keeping the XML node-id of an assertion with an RDF triple context could be a great way to prevent RDF triple bloom for context.</span></p><p class="MsoNormal"><span class="entry-content">Finally, I felt that David Wood and James Leigh's presentation on </span><span class="status-body"><span class="status-content"><span class="entry-content"><a href="http://code.google.com/p/callimachus/">Callimacus </a>project had the most potential to have a "big impact" on the adoption of Semantic Web tools. This new open source project allows users to specify a simple HTML template with special XML tags that allows them to bring triples directly into a web application. This has the potential to allow far more non-SPARQL programmers to integrate RDF data directly into a web page much like XQuery does today. With some work it might be possible to auto-generate the XForms bind elements for form rules. This would be a huge win for the integration of rules into web forms without needing to write JavaScript.
</span></span></span></p><p class="MsoNormal"><span class="status-body"><span class="status-content"><span class="entry-content">You can also see my tweets on the conference here: http://twitter.com/dmccreary</span></span></span></p><p class="MsoNormal">I hope to see more of you in San Francisco next year. Let me know if you are interested in co-presenting any papers!
<span class="status-body"><span class="status-content"><span class="entry-content"></span></span></span></p>Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com1tag:blogger.com,1999:blog-13546516.post-5992352697818040232008-11-19T15:06:00.001-06:002008-11-19T15:22:29.124-06:00Problems with WebDAV on Vista 64bitDoes anyone have any suggestions on a good reliable WebDAV product for Vista 64bit?
I can not get the one that is bundled with Vista to work. Every time I try to create a remote connection I get the following error message:
<br/><br/>
"The folder you entered does not appear to be valid"
<br/><br/>
I have run the Vista WebDAV patches:
<br/><br/>
<a href="http://www.microsoft.com/downloads/details.aspx?familyid=17C36612-632E-4C04-9382-987622ED1D64&displaylang=en">KB907306</a><br/><br/>
I have also tried the following Hotfix:
<br/>
<br/>
<a href="http://hotfixv4.microsoft.com/Windows%20Vista/sp2/Fix224785/6000/free/347160_intl_x64_zip.exe">Windows Vista WebDAV Hotfix</a>
<br/>
<br/>
I have also tried to get various 3rd party packages working such as Independent DAV, BitKinex and WebDrive.
All of the WebDAV products seem to have problems with the Windows Vista 64 bit systems.
Is there any way to get Vista to use the Windows XP WebDAV "drivers"?
I am using both eXist and MarkLogic native XML databases.
Thanks - DanDan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com1tag:blogger.com,1999:blog-13546516.post-9508728083597352762008-11-12T11:48:00.003-06:002008-11-16T12:23:44.848-06:00How to compile the eXist 1.3 buildI have been learning how to use higher-order functions in XQuery. I am using the eXist system to test this.
To do this you will need to get the 1.3 build that there is no download for (yet).
Here are the steps I used:
<ol>
<li>Download fresh copy of eXist from svn via TortoiseSVN or Eclipse subversion subclipse plugin from the eXist SVN sourceforge repository:
<pre>https://exist.svn.sourceforge.net/svnroot/exist</pre>
</li>
<li>(Confirm environmental variables are set correctly for JAVA_HOME and EXIST_HOME) To do this I use the Windows/Computer/Properties/Advanced Settings and check it with the SET command at the CMD prompt. The result looks like this:
<pre>
JAVA_HOME=C:\Program Files\Java\jdk1.6.0_10
</pre>
My Eclipse workspace is just "C:\ws" and I created a project called eXist-1.3 so the path name I used for EXIST_HOME is:
<pre>
EXIST_HOME=C:\ws\eXist-1.3dev
</pre>
</li><li>Open the DOS cmd prompt, cd to eXist directory
</li><li>type in "build.bat"
</li><li>Type in "build.bat -f build\scripts\jarsigner.xml"
</li></ol>
It took just 1 minute and 6 seconds for the main build to run on my 4CPU system with 8GB of RAM even though it is running Vista.
Then I just ran the start.bat in the bin directory. eXist was then running on http://localhost:8080/exist. I had to use the new WebStart admin tool on the admin section of the eXist web page to change the admin password.
Thanks to Joe Wicentowski at the US Dept of State for helping out!Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-53404556387009477242008-06-05T07:34:00.001-05:002008-06-05T07:49:05.215-05:00The Finch and the RaccoonThe blogs on xml.com are down so I have reposed this here.
Have you ever wondered if the laws of evolution apply to computer languages? When you walk down the isle at your favorite bookstore, does it seam like there are actually more computer languages than last year? What forces are driving each of these new languages to evolve?
In 1835 Charles Darwin visited the Galapagos Islands. There he collected what he thought were about a dozen distinct species of birds. Upon returning to England he discovered that each of these species had evolved from a single species of finches. On the various Galapagos Islands the requirements for food gathering was different, but consistent over hundreds of thousands of years. Enough time for a single species to adapt to meet consistent requirements.
Consider the Raccoon: omnivores that have proved to be one of the most adaptable mammals on Earth. The Raccoon’s range has rapidly expanded into urban areas due to their ability to quickly adapt to new requirements before other animals have had time for the wheels of evolution to turn.
So goes it with computer languages. Some procedural languages can be quickly adapted to fill in the needs for a new niche. When the web was young, procedural languages like Java and JavaScript quickly filled in the need for a variety of tasks. As the requirements for building web applications stabilized, declarative systems like CSS, XForms and XQuery started to push procedural languages back into niche-areas. As these declarative languages stabilize and become worldwide standards, graphical tools are being created to allow non-programmers to create, manipulate and extend these systems.
This is why many of us believe their will always be some need for procedural programming, but certainly not for building standard web applications that are controlled by style sheets and user interaction forms. Like the finch, declarative languages need a little longer to evolve. It sometimes takes years for a small vocabulary of functional specification patterns to emerge and be given labels. Additionally, it can takes years for the standards bodies to agree on the best way to deliver these new languages in a set of semantically precise data elements that have unambiguous interpretations. Finally, it may take another few years for IT managers to realized that they really do lower costs if they avoid vendor-specific implementations and adopt worldwide standards.
When CSS first came out you may have been a little reluctant to let web designers play with a rules engine. As XForms becomes ubiquitous you may be resisting change because you have invested so much time and energy learning how to debug JavaScript (without a debugger). You can not hold back the forces of evolution…and now we all need to adapt to the declarative world or risk our own extinction.
If you are interested in more on this topic see my Presentation from the 2007 Semantic Technology Conference <a href="http://www.danmccreary.com/presentations/sem-web-07/">The Semantics of Declarative Systems</a>Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0tag:blogger.com,1999:blog-13546516.post-24492401973368261522008-05-27T07:33:00.001-05:002008-05-27T07:41:25.795-05:00XRX: Simple, Elegant, DisruptiveI recently started writing for O’Reilly Media. I posted an article on XRX.
Here is the link:
<br/>
<a href="http://www.oreillynet.com/xml/blog/2008/05/xrx_a_simple_elegant_disruptiv_1.html">http://www.oreillynet.com/xml/blog/2008/05/xrx_a_simple_elegant_disruptiv_1.html</a>
<br/>
We are just getting started moving to a new MoveableType (MT) system so not all of the features (like keywords and feedback) are working.
<br/>
If you have problems commenting on the site, please feel free to post your comments here.
<br/>
Here is a comment from Arun Batchu:
<br/>
<em>An elegant introduction to an elegant architecture, Dan. Thank you. The essential takeaway from what you describe is the exploitation of XML from one end to the other end - especially from the Developer's perspective, for, how XRX actually manifests in runtime could be left to implementation technologies. Thus the logical architecture of XRX could be realized by a few variations of concrete technology - which is great. The XForms could be realized by XForms server technology (such as the excellent Orbeon stack), the ReST could be realized by any middle tier and the XQuery could be realized by an XQuery engine (such as Data Direct) that may actually be driving any one or a combination of datastores (XML, SQL, file system or ...) . The symmetrical and consistent leverage of XML as a data model from creation to transport to rest and back eliminates a whole lot of wasteful work. Like you point out, XPath is one of the most powerful query systems I have encountered; you can pack so much in so little and reuse it across the board with little if any change from the drawing board to production. In such a system as you describe, a business rule expressed once can be reused anywhere - from one end-to-end , however long the travel, as long as the architecture is XRX, like you have described. Thanks for expressing it so well. A few of your readers will not get it - it is one of those things that once you experience it, you are left wondering why this did not happen before. Oh, well!</em>
<br/>
<br/>
You are welcome! Thank for your feedback Arun!Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com1tag:blogger.com,1999:blog-13546516.post-4444034011876705752008-03-22T11:34:00.000-05:002008-03-22T11:40:11.066-05:00XForms, Dyslexia and the Right Brain<p>Those of you that have worked with me know that I am a little bit of an odd-duck. I have dyslexia. If you ever see me attempt to write on a whiteboard my spelling is at the ninth grade level. My left brain, used for phoneme recognition, never really developed like the rest of you. My right brain had to be co-opted to help out. But one of the skills I seemed to have picked up due to my over-exercised right brain is the ability to visualize multiple complex application architectures and quickly understand architectural tradeoffs. I am one of the few people that seem to be interested in discussing how XForms, metadata registries, ontologies, the semantic web, OWL, RDF, graphs, business rules, BPM and Kimball conformed dimensions all can work together to deliver elegant and cost-effective enterprise-scale solutions. It seems easy for me to simultaneously visualize two or more architectures and it constantly challenges my patience when I have to explain over and over why architecture alternatives will not meet a business requirement.</p>
<p>It turns out that many people that have dyslexia also have the gift of being able to visualize complex systems. Albert Einstein, Thomas Edison, Jackie Steward and Charles Schwab are good examples of dyslexic people that have used the strengths of the right brain to do things that left-brain thinkers could not.</p>
<p>As a right-brain centric person I need to also tell you that I really love the XForms architecture. The magic of a declarative language, MVC, bindings and a dependency graph makes XForms development 10 years more advanced than anything else I have worked with. I think it is beautiful and elegant. It is everything that AJAX and JavaScript application are not. Clean, simple and easy to visualize (for me at least). When someone asks me if I can create an XForms application to do something, I create a mental image in my mind of the model, the view and how events will update instance data in the model using inserts or external submission results. I can easily visualize the bindings of view controls to data elements in the model. Once I can visualize the application clearly, writing the application is just a matter of typing in the code.</p>
<p>I think that my right-brain is also a reason I detest JavaScript and AJAX. It is far too much code to read and trying to visualize how 300 lines of JavaScript enables me to do a drag-and-drop. I want to just add an attribute to an element like "drag-source" and "drop-target" and I want it to just work.</p>
<p>What triggered this posting is that I have been reading <i>Proust and the Squid</i> by Maryanne Wolf. This is a book about how the brain's circuits are used in the reading process. She has a wonderful explanation of how the dyslexic brain co-opts the right brain for reading and enhances it functionality. I didn't really understand the relationship between my defects and my gifts .</p>
<p>So how about you and your development team? Do you have a dyslexic right-brained person on your team? Can they quickly visualize architectural tradeoffs? Have they tried XForms? And if they do, will you be willing to tolerate their disgust of AJAX and JavaScript after they have built their first XForms applications?</p>
For more information about dyslexia check out these two Wikipedia entries:
http://en.wikipedia.org/wiki/List_of_notable_people_diagnosed_with_dyslexia
http://en.wikipedia.org/wiki/DyslexiaDan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com2tag:blogger.com,1999:blog-13546516.post-70890489780630402782008-03-21T15:01:00.001-05:002008-03-21T15:55:02.964-05:00Great Example of Multi-dimensional Bubble ChartHere is a beautiful example of a bubble chart display using global population statistics. The example on carbon emissions are very interesting.
Note the dimensions
<ol>
<li>
Population of County (size of bubble)
</li>
<li>
Continent (color of bubble)
</li>
<li>
Live Expectancy (vertical axis)
</li>
<li>
Income (horizontal)
</li>
<li>
Time (the play button)
</li>
</ol>
It is interesting to see the huge impact AIDS has had on the life expectancy in African counties.
Imagine if you could see your organizations product sales using this type of graph.
This application was done with a software system called Trendalyzer. It was initially developed by Hans Rosling's Gapminder Foundation in Sweden and acquired by Google Inc. in March 2007. This version is a Flash application.
Does anyone know of any open-source software that could do this?Dan McCrearyhttp://www.blogger.com/profile/12453673235365396446noreply@blogger.com0