Friday, October 29, 2010

TangibleStories Brainstorming

For HW2, I have to design a tangible interface that allows people to share photos, videos, and text. I am using the Microsoft Surface SDK to construct this tui. 

- at least four different surface controls
- two physical objects (tags)
- display photos, text, video 

Hand in
- project files
- readme file (Reality-based interaction, Tangible Interaction themes)
- photos of pyhsical objects
- 30-60 seconds video 

User scenario:Tupelos (Wellesley College a cappella) members are on a Tupie retreat. While hanging out, they bring up TangibleStories to look at videos, pictures, and lyrics of songs they sang and talk about great memories from the past. One of the members picks up a Musical Note tag to play a list of videos. They listen to performances and look at the choreographies; Tupies pick up the dance moves and entertain the other members.

Important Notes
Reality-Based interaction:
- Naive Physics
- Body Awareness & Skills
- Environment Awareness & Skills
- Social Awareness & Skills
Source: "Reality-Based Interaction: A Framework for Post-WIMP Interfaces"
by Rober J.K. Jacob, Audrey Girouard, Leanne M. Hirshfield, Micael S. Horn, Orit Shaer, Erin Treacy Solovey, Jamie Zigelbaum

Tangible Interaction Themes:
- Tangible Manipulation: tactile qualities
- Spatial Interaction: movement in "real space"
- Embodied Facilitation: material objects, space's effect on group behavior
- Expressive Representation: material and representation expressiveness and legibility
Source: "Getting a Grip on Tangible Interaction: A Framework on Physical Space and Social Interaction"
by Eva Hornecker, Jacob Buur

Friday, October 22, 2010

Tangible Kindergarten for Children by Professor Marina Bers

Associate Professor Marina Bers from the Eliot-Pearson Department of Child Development and the Computer Science Department at Tufts University visited Wellesley to give a talk on Tangible Kindergarten for Children. She emphasized the need for education to include a curriculum centered around robotics and engineering to insure that kids, who nowadays grow up surrounded by machines and advanced technology, learn and understand their interactive space. Because society has changed to an extent we often find ourselves depending so heavily on daily tools and gadgets we can't individually construct, it is essential that students are taught to at least understand the concepts behind the complicated manufacturing in our surrounding.

The interviews and video clips on the performance of children who experimented with the robotic kit were a fascinating testimony to how engineering can be an intuitive and natural part of learning. Kids, in the short period of 20 hours, were not only able to build and program robots but also integrate their new learning with their life stories. Computer science is no longer an extension from math and physics, but it rather becomes another fundamental study in life that will help people engage with the way their society is literally constructed. I was really impressed and even jealous of the length of exposure these children had to the world of technology that I am only beginning to understand. It almost seems now that such advancement in the education system is an inevitable part of change.

I am curious to know what more changes are being implemented in the kit to improve the transition between GUI and physical toys. I certainly do not want kids to be overpowered by computer screens. They should be taught to take control of their learning kits and learn to summarize the bigger picture of the tools they are exposed to.

VideoPlay: Multi-Touch Video Editing

While exploring through publications from Microsoft Research, I came across another TUI design for video editing. Stuart Taylor, Shahram Izadi, Kursat Ozenc, and Richard Harper devised VideoPlay, a TUI that functions on the Microsoft Surface platform and allows interactive manipulation and editing of video clips mainly from mobile devices. This device encourages fluid transitions and collaboration between physical objects, such as physical tiles that can map a video on the surface and connect with each other, and the digital effects that execute many of the editing process.
The tiles are designed to lock with each other to signify transition, and this action is read by the surface to produce a preview. The user can also attach tiles that symbolize certain effects literally on to the video clips. 
The user can also directly manipulate the videos with touch. By sliding a tile while holding on to one end, the video unravels into a sequence, and the user can literally crop scenes by slashing one's finger through them.

VideoPlay successfully integrates the touch manipulation provided by a tabletop and the physicality of tile objects to produce an intuitive video editing system. I am interested in learning more about the TUI; for instance, I'm curious what kind of effects one can add to the videos. Overall, VideoPlay provides a beautifully designed tangible surface that is intuitive to users. I hope to see more development on the project such as additions of sound and visual editing.


Thursday, October 21, 2010

Moritz Waldemeyer

Moritz Waldemeyer is a contemporary designer who explores and combines fashion and technology to construct beautifully engineered art works. Here are some images of his work. This will serve as factors of motivation and inspiration for the aesthetic aspect of Bubble Pop Electric.

Airborne video dress carrying 15,000 LEDs (wow) and displaying video sequences (collaborative work with Hussein Chalayan)

One Hundred and One (collaborative work with Hussein Chalayan)

Rhianna's Laser Outfit for the Grammy's


Waldemeyer's design works not only signify the beauty of skillful manipulation of light engineering but also promotes practicality of these futuristic products. They sometimes serve as extremely attractive and alluring performance outfits and other times become modes of effective communication and advertisement. His products are a true inspiration for the team, and I wish to concentrate more on the aspect of appearance as a superhero also looks good in his/her outfit.

Source: links found at


Alex from the Bubble Pop Electric team suggested Pygame to program the music suit, so I explored a little bit with the coding.

Pygame provide python modules that can be constructed to create multimedia games. The music module provides a set of commands of audio that could be useful for reading the manipulations on the suit. return None 
This command will load the audio indicated. This action should be executed when a music bubble is turned on. However, the drawback is that it will stop the clip already playing, and we need a system that can play multiple tracks at once. return time
The get_pos command returns the time the music has been playing for. This value is useful to position additional music clips that will have to sync with the music that is already playing. For example, if the tempo of the music is indicated to have measures that are 4 seconds each, the additional clip that is activated by the player can be programmed to come in at the nearest time characterized by multiples of 4. If the user plays a beat bubble while a main song is playing, the computer calculates the main music's time by using: return time

If the return value is divisible by four, the beat music will play immediately through, start=0.0): return None

If it is not divisible by three, the computer retrieves the value of the remainder, subtract that number from 4, wait for that many seconds and then play the clip.

This scenario is highly theoretical, but this study will hopefully help me visualize the programming aspect of the TUI.


Building LED Shirt Reactive to Motion: Using LilyPad

Leah Buechley from the MIT lab gives detailed instructions on creating accelerometer t-shirt with LED lights using the LilyPad Arduino kit. The LilyPad provides basic tools for building wearable electronic clothing, so I believe it is worth studying one of Leah's tutorials. Our team is planning on building a LED panel that reacts to outside or internal influences, so tutorial will hopefully give me a good sense on construction based on sensors.

These are some of the materials needed for the construction:
LilyPad Arduino Main Board, power supply (we are planning on using batteries that are going to be part of the stretch suit), USB Link, accelerometer, conductive thread, LED, digital multimeter, needle, fabric paint, fabric glue, needles, puffy fabric paint, etc.

1. Stitch the LilyPad and the power supply on the garment (make sure they remain in close distance for strong connection).
2. Mount the accelerometer and LED (seal knots with fabric glue).
3. Insulate power lines with puffy fabric paint (we may just attach another layer of fabric on top of the LED panel to create an illuminating effect).
4. Program the Arduino software (my team may program the LilyPad to react to the music playing).
The actual construction of the LED panel, with the right materials, seem simple and intuitive. The team's main concern should be the programming aspect of the LED panel.

Some ideas for programming
- LED reacts to the bass of the music
- LED reacts to the loudness of the music (using sound sensors)
- LED generates graphics pre-programmed for each beat sequence or main music

In order for the LED panel to generate graphics, each LED needs to be constructed into a grid system or an array. It needs to be identified based on its location, and each LED will be instructed independently.The problem with this task is that each LED would have to be connected to the LilyPad separately, and I'm afraid it won't provide sufficient ports. Also, translating the sensors may be a challenge because reading music requires skills to identify amplitude and timing.


Thursday, October 14, 2010

P1 Conceptual Design Summary

Use scenario:
Stefani is a young DJ/entertainer, and she is looking for a new way to present music. She decides to wear Bubble Pop Electric to a party. She enters the stage-nervous. She switches one of the main beat bubbles. She twists it to adjust the volume, and she then turns a main music bubble to start the main song. The LED panel on the body suit displays animation following the music. As the crowd gets excited, she turns on her beat shoes and begins stomping. The stomp shoes accelerate the bass of the music every time the shoes hit the ground, and the LEDs on the shoes react to her stomping. Stefani feels more comfortable on stage, so she begins dancing. The suit reads the capacitance and creates a whammy bar effect on the music. With LEDs flashing all over the body along with Stefani's cool music mash, the crowd is more excited than ever. 

Storyboard: This is Stefani's story

The sketch accommodates changes made based on class discussion. The team decided to eliminate the hat due to its lack of tie to the main suit and a clear purpose. Also, instead of providing a sash as an active space, we decided to keep the bubbles in designated areas. This reduces the danger of bubbles falling everywhere and confusion over the location of bubbles. The area where the sash is to be replaced with a LED panel that will display visuals based on the music playing. We are also planning to add a belt with on-and-off switches+sliders for the beat shoes and the LED panel. Overall, the main compositions of Bubble Pop Electric includes:

1. Stretch sensor suit+LED panel: the stretch sensors located on the sides will measure the capacitance of the user's movement and create whammy bar effects according to the movement. The LED panel will display animation based on music playing.

2. Music bubbles: The music bubbles are designated in four different areas of the body-right and left shoulders and right and left hip/thighs. Each designation carries a main, beat, and sample music and you may turn them on my twisting them. Once activated, the LED lights in the bubbles will light up.

3. Beat shoes: The beat shoes are embedded with vibration sensors. They will react whenever the user stomps. This will translate into acceleration of the bass and also LED lights turning on. 

(Also, there will be a belt to turn the devices on and off.)

Low-fidelity prototype:
This is the video of the demonstration we did in class with our low-fidelity prototype.

And the following is a screen shot of the GarageBand composition used for the demo.