Implementation 1 - java based rhythmic listener and interpreter

All 4 steps performed by a java programme

Java was selected since it allows one programme to run independently of the platform, Windows, OSX, Linux.

A

The user can perform a rhythm using up to 5dedicated keys on the keyboard. B, J, K, L, M were selected since they fall easily enough under the fingers of a right handed user. Entries are performed in a way close to playing keys and chords on a piano keyboard.

B

keystrokes on the dedicated keys are converted into "chords" and "phrases"
"Chord" represents the keys that are pressed simultaneously - it is basically an array of 5, with values 1 or 0.
"Phrase" represents a group of chords separated by a variable interval. This is the actual rhythmic pattern.
The length of the interval between chords is recorded using a number. The shortest interval is given the value 1, the other intervals are given a multiple (2 if the interval is twice as long as the shortest in the phrase, ...)

The first prototype of the code actually uses a string to record the chords and phrases
example of chord : _o_o_ : the second and fourth key are pressed
example of phrase :
o____1o____1o____ : first key is pressed 3 times at regular interval.

C

The phrases are then compared to a "dictionary".
The dictionary is created by the user and consists in a list of couples (phrase,translation)
If a translation is found in the dictionary for a pattern, the translation is displayed and interpreted

D

Interpreter is a piece of code that takes as input the translation and looks at its structure to try to convert it into action.
Current translations at this stage are as follows :
print a string (text)
send a key strike or combination of keystrokes (performs keyboard shortcuts)
open a programme or file (tested with "mail")

Download the programme (v3 - jar file) on this link : jar file

How to use the rhythmic programme

The programme uses the keyboard keys as shown on the illustration below as input for the 5 channels.

This set up works for a right handed user using 5 fingers to enter a rhythm in a similar way to playing 5 notes on a piano.

When opening the programme, a window is displayed, with 4 areas. Make sure the focus is on the input box and start typing your rhythms.
Type in the succession of inputs fast enough to have them recorded as a single pattern. After a time period without and entry (around 0.5 sec), the programmes considers that the phrase is ended.

The phrase is displayed in the left pane.

The translation is displayed in the right pane.

The mode button toggles from "read" mode to "learn" mode.

alternatively from pressing the button, mode command is available by default with the rhythm : o____1_o___1__o__1___o_1____o

Read mode

This is the default mode, as described above. Rhythm is entered, translated, interpreted.

Learn mode

In this mode, if a rhythm entered is not recognized (ie does not yet have a translation), the user is invited to type in a translation.

To procede without saving a new translation, just press RETURN, and start on a new rhtythm.

To go back to "read" mode, press the button or type in the rhythm combination.

Translations

Any text can be registered as a translation
The text is displayed if found

Interpretations

Some commands are accessed through special words :

to date, v3 has the following special commands :

$mode : toggles the mode from read to learn
$beep : plays a sound from the computer
$key_F9 : sends keystroke F9 to the host computer. This type of action ca be developed to send any key combinaison
$mail : open a new email with the default email programme - this type of action can be developed to open any file or programme

Managing the user's rhythms dictionary

The programme stores the dictionary in a text file called "data". On the first opening the file is created with a few default rhythms and translations.
The user can either edit the file or develop it using the learn mode describe above.

Comments

Processing - playing with particles collections

Colourful ant-like particles description Particles created, move, interact, grow from encounters and disappear. click and drag to create new particles Processing code: Automated vehicles reaching target and avoiding obstacles description Processing code: Coil spring description Processing code: Spring 2D description Processing code:

Implementation 3 - use arduino uno and bluetooth link

concept: Function A (input) performed by an arduino uno connected through bluetooth as using a serial communication. Other function B C D actions performed by a java programme. Image of hardware setup: Arduino with switches, bluetooth adapter, serial link The main benefits of this configuration is that the keyboard is not needed anymore as a link in the input chain. The host can then listen to the bluetooth data sent in the background, while the keyboard is being used for different purpose. Also, the application on the host does not need anymore to have the focus. Last, the input device can be split from the host computer or device.

Processing - drawing interface good for kids

Drawing interface using processing 3 Description This sketch allows to draw shapes simply. Click once to activate the pen. An initial color is selected. Press an arrow button to select color and mix cyan, magenta, yellow, black using arrows. Press SPACE to get back to drawing. Draw as long as the pen is activated moving the mouse. Then click again to deactivate the pen. Each line can be erased by pressing SPACE. Erasing takes place in the reverse order of it being drawn. code Processing:

Arduino projects, sensors and actuators

Moving average on analog input Description This small project shows how to implement a moving average with a lightweight form in order to smooth raw analog values. The value is then used to position a servo Image code Arduino: Using Ultrasonic sensor Description In this project, we are using a US sensor. Very first step to understand the behaviour of the sensor through the serial interface. Image code Arduino:

Home Automation - Garage Door Security

Description: This is the research project to understand the home automation system which I would like to develop when I'm building my own house in the future. Burglary is a common threat while living in South Auckland. My wife's mountain bike was stolen from the garage in our first year living in the South Auckland. Even though we have moved to different house and living in better neighbourhood now, always worried and paranoid about the garage door. So this project developed to monitor the garage door and alert if the door is open. Some of the terminology used in this post: Home Assistant - Home Assistant is an open-source home automation platform running on Python 3. Track and control all devices at home and automate control. Many components are supported well by Home assistant community (Wemo, Amazon Echo, Google Home, Google Assistant, Zigbee, MQTT, IFTT, etc..) MQTT (aka MQ Telemetry Transport) is a machine-to-machine or “Internet of Things” conn...

Drive your lego model by voice

Voice controlled Lego merry-go-round Build a model using lego and a motor Drive the motor at various speed using an arduino board as variable frequency drive Send commands to the arduino board using your voice, through a Processing sketch on your computer and bluetooth That's it, and it works Description hardware A Lego motor - old motor from Lego technics - DC current - 4.5V Arduino - Uno in our case Bluetooth board - ref Transistor Diode Breadboard and jumper cables Arduino code : Processing code :

ABC game - letters sounds and rhymes

Audio interactive game for kids around letters and numbers learning Description Toddlers are attracted to computers, they want to play with it, press the keyboard all the time when their parents use it - pretty much for everything. I was browsing the web for games designed to help kids learn their letters and ABC when I decided to take the opportunity to build a simple game allowing the child to discover the letters, while taking an active role in building the game it self. Here is the concept: The game displays a letter and at the same time plays an audio relating to the letter. If the player presses the right key at the first attempt, a congratulation sound is played and lots of progress is made. If the right letter / character is found after several attempts, less progress is made, no congrats sound is played. The child or the parents can record every sound matching the letters as well as the congrats messages. I got my daughter to record a short voice memo to ma...

Smart temperature sensor and client interface

Wifi enabled temperature and humidity sensor and client sending email notifications description This project was developed as part of a research to build a temperature automation system for our charming but cold Auckland home. Relying on electric oil heaters as sole source of heating in children's room, the aim was to monitor continuously the temperature at night and to turn on or off the heater according to the measured temperature. The project described in this post does the first half and mode demanding part of the job which is log a temperature, broadcast it to the web using the MQTT protocol through the house's wifi. A small and cheap ESP8266 Wifi board programmes using the Arduino language reads temperature and humidity through an DHT22 sensor and broadcasts the values to an MQTT server at set intervals. A custom client written in the java based Processing 3 and running on a computer located anywhere (Macbook in this case) displays the readings and sends not...

Rhythmic interface with machines

Introduction Many forms of human user interfaces are available to interact with electronic devices. Most commons ones use a keyboard and mouse and touch screens. Voice interface is developing and gaining momentum. This project was started from the observation that most commonly used HID require that the user has a visual contact with the interface device (applies in particular with touch devices like smartphones and tablets, also with keyboards). Some common or recurring tasks however could be completed without a visual contact with the device if the interface was designed for that. These task include launching a specific app, opening a file, writing a standard text... The project will explore an ancient and yet rich, not to say endless means of communication which are rhythms. Drums have been used to carry messages between villages. Some people living in rural areas use song and whistle as a second specialist second language to communicate on a longer dist...

Manukau tech projects

Search This Blog