nuSense is a degree project on master level in interaction design from the Umeå Institute of Design. The project focuses on wearable technology, why innovation is slow, what we can do to change it and in the process extend our senses.
As this project is substantial in length please use the menu below to navigate, and the small up-arrows next to the headings to return here.
Wearable technology is everywhere, you can hardly walk down the street in most cities without seeing someone with a fitness tracker on their arm, or some sort of glasses which augments your view. While wearable technology appears to be running amok and new devices are released almost daily there appears to be a void beyond the fitness trackers and the glasses. When was the last time you saw a wearable device that was not one of those? They do exist, devices like Narrative Clip for instance, but they are in vast minority. Wearable technology is by and large still stuck in the place iPads were when they first came out, we’re still asking ourselves what to do with it. Fitness trackers can’t be the end goal for wearable tech, can it?
We should maybe also ask ourselves what wearable technology, and to an extent any technology we carry with us, are doing with the way we interact with the world. They alter our perceptions and give us insight into data which we did not know about ourselves, at least not to the fidelity they provide. In a way we have always altered our perceptions in one way or another but this is the first time in history where that leap has really reached far. We have entered the realm of augmenting and creating new senses.
This degree project deals with these two exact concepts. Why wearable technology has reached somewhat of a creative slump, and how we can further extend our senses through wearable technology.
Ever since the dawn of time we have tried to improve our situation by using tools, and those tools change and improve our abilities. The stone made it possible to crack shells open, the wheel made it possible to travel faster and smoother and the transistor made it possible to compute things at breackneck speeds. Many of these improvements have dealt directly with how we perceive the world, when we couldn’t hear we made funnels and later hearing aids, when we needed to see further we invented the telescope and binoculars. In a way even the mobile phones we carry with us alter our senses in how we navigate around our environments, we’ve effectively outsourced some of our brain power away from dealing with navigation to a small box with a GPS app in our pocket. In doing so we have also added capabilities that we did not have, like traffic information, or places of interest.
What’s to stop us there though, why not continue making improvements to our senses? Why stop at the senses we already have, when we could even make new ones?
Sensory augmentation and wearable technology go very much hand in hand as they’re both close to our beings. The difficulty comes though when prototyping and building for wearable technology. A stationary device can take almost any form and still be functional, but the form and the function of something that exists on your body and needs to be adapted to different body types becomes infinitely more complex. There is a huge hurdle to overcome in developing wearable tech, and as such a costly one as well. This may be one reason why we don’t see much variation in wearables, why one armband is much like another, one eyeglass like the next.
There are however those who have experimented in this realm already, people like Neil Harbisson who was born entirely color-blind but built himself an electronic eye which allows him to perceive color through audio frequencies played to him through headphones. There’s Steve Mann who by some was dubbed as the “father of the wearable computer” who has been wearing one version or another of his EyeTap device since the 1980’s. What they have in common is that they’re both enthusiasts and are using pieces of highly personalized hardware and software to do what they want. Their material remains inaccessible for the average user.
So what can we do to make it easier to prototype wearable technology, to create new senses and to make the experience as unrestrictive as possible?
One such way may be through modular technology. It’s an idea based on creating complete systems in parts, which are in turn interchangeable. It’s nothing new or revolutionary, modular thinking has existed for decades, not the least in architecture where kit houses can be constructed elsewhere, delivered and assembled on place. More recently we’ve seen modular devices enter the electronics field, devices like Google’s Project Ara which promises to be the last mobile phone you’ll ever buy just because of its modular nature. There are also electronic prototyping platforms, like littleBits and Cubelets which offer their take on building electronics easily without having to deal with running cables, soldering or programming.
During research a survey was carried out where users were asked to answer to what extent they already consider themselves augmented, in what way, and how open they were to wearable technology and where their boundaries laid. While the answers were skewed somewhat towards the western world and towards people in their 30s some interesting data could be gathered. Many people have a rather loose idea of what wearable technology and augmentations are, and some even listed items of clothing as things they considered as augmenting themselves. The really interesting thing was however that 87% of the respondents said that they would be open to trying or using sensory augmenting wearable technology. When asked what they would like to sense their answers were rather varied, such as zoom lenses for eyes, super hearing, having a ”calm button” to one person even expressing interest to be able to taste email notifications.
Dimensional Innovations: Wearable Technology Lacks Innovation.
Four things have become clear coming out of research.
2. Wearable technology has under-delivered on expectations. Research and development is expensive so making safe devices is financially sound. Consumers may buy the next device, and the next after that, but if each new device is essentially the same as the one before, and as the competitors device, consumers will lose interest.
4. Modular technology is a well explored field, and one that holds great potential for both prototyping and growth. Yet it is a field that has been left untouched within the area of wearable technology.
So what can we do to open up the field of wearable technology and make it blossom beyond armbands and glasses? At the moment hardware prototyping is predominately geared towards stationary devices. It’s easy to get an Arduino and start building something which lives in a single place. However, there is a lack of a proper platform geared towards wearable technology.
A few wearable exploratory prototypes were built to better understand sensory augmentation, as well as the challenges of prototyping for wearable technology.
The first prototype puts a homage to the biohackers, those who are interested to truly augment your physical body through technological means, by super gluing a magnet to the tip of a finger. In reality biohackers implant the magnet under the skin and through that can sense electromagnetic fields around them through vibrations in the magnet. While gluing it to the surface of the skin lessens the sensitivity my some magnitude it was still plenty sensitive to notice electromagnetic fields around electronics. Interesting, if novel, way to augment your senses and add something you weren’t able to sense before.
The second prototype focused more on augmenting your sight, and was a sideways periscope of sorts. It was a mask with two mirrors inside that fit over your eyes and directed your vision behind you. While it was entirely possible to function with the mask on, left became right and up became down and nausea started setting in before long. For the safety of myself and those around the experiment was cut short. Though important to note that when you disrupt your senses too much the experience becomes unpleasant to say the least.
The third and final prototype explored the realm of electronics in wearable technology. It was a hand mounted prototype with a distance sensor and a vibrator. The distance sensor triggered the vibrator differently much depending on how far away objects were. The result was a crude depth sensing making it possible to navigate spaces even when you couldn’t see. One interesting thing was how quickly this new sense became second nature, and you stopped actively reacting to it, but rather moving your hand instinctively.
The difficulty in designing for sensory augmentation using current technology shows that there is much that can be done to make sensor technology more accessible even for those already adept at electronic prototyping, but maybe especially for those who are not.
Who would benefit from being able to prototype wearable technology in an easier way though? This project primarily aims towards three groups of users, each with their own set of needs, but each who would benefit from it in their own way.
3. The third group are the aforementioned bio hackers, those interested in physically hacking into their bodies with electronics. As of now it’s very hard for them to get any work done, largely because no medical professional want to help them out due to the Hippocratic oath. While they’re a bit of an outlier in this project being able to prototype abilities and augments which they would like to make part of their bodies would at least help them to go part way to their goal. It would also give them the possibility to refine their augments before actually implanting them into their bodies.
The project, and concept, is intentionally left broad and somewhat open ended to allow users to find and create their own purpose through the tool. Its purpose is not for a user group to use the result of this project, put it down and move on, but rather to create their own purpose, and to have something to build upon.
For a development platform to become successful you need more than just hardware of course. Taking Arduino, undoubtedly the biggest and most well known hardware developing platform out there, as an example you definitely need two more things: an easy to use IDE and an active community. Without any of these three legs the proverbial chair would fall over, and a concept built on such would also not work very well. The hardware needs to be in place, the IDE easy to use, even for those who are not programmers and the community needs to be in place to allow users to help each other out and to push the possibilities of the platform further.
The focus of this project has been to connect these three pieces together, rather than to design each piece in isolation. While at its core it’s about a platform for prototyping wearable technology and augment senses it’s also a system with interconnected parts.
One may believe that the field of wearable design already is one that’s already fairly defined, with set rules and guidelines. Unfortunately, maybe because we are so dissimilar as people, that’s not quite so. There has been research done to try to ascertain optimal placement for wearable devices, like the illustration above from Carnegie Mellon. However, are we’re making a platform to build prototypes such data may at best only help to guide further guidelines, but not necessarily to act as a template for design. In fact, since it’s a prototyping platform the user should remain as free as possible in their decisions, and that includes the freedom to make mistakes. Even if it means building something stupendously impractical it should be up to the user to choose to do so, not to the platform to restrict them from doing it.
Another consideration that may be more important is one of form and material though. As wearable technology is close to the body some sort of ergonomic requirements should at least be considered. It would be great if we could design for everyone, even the 2.30m tall, six-fingered, outliers, but for sanity’s sake it may be better if we stick to some sort of standard deviation in relation to body type.
The question really boils down to identity and practicality. One may think that making the smoothest possible design will be best for the human ergonomy, but then you run the risk of designing something which looks more like a baby’s chew toy than anything else. Go too far in the other direction and things become too edgy and uncomfortable. The golden spot lies somewhere in the middle with well defined shapes but well rounded corners.
nuSense comes as a kit with everything you need to get started creating and experimenting with wearable technology. In the kit you get the basic building blocks, a few sensors, a few actuators, the base unit, cables and battery. There’s also a small quick start guide to help you take the first steps to getting nuSense up and running, and having a few example projects to start out with.
The sensors are mainly based on the senses we already possess, a distance sensor, a direction sensor (compass), an acceleration sensor, sound sensor (microphone), light sensor and a heartbeat sensor. With these sensors you can already reach a fairly broad spectrum of the world around you, and allow you to tap into some things which are available, but not easily accessible, such as telling the direction you’re pointing, or sensing your movement with a higher granularity.
Just as you get six sensors, you also get six actuators, also following the line of the sensors with being based on triggering the more basic senses. It has a vibration motor, a speaker, a pushing actuator, for poking or tapping you, a small screen for visual input, a heating pad and a motor.
To mount the individual modules we could have designed an elaborate mounting system using proprietary, or at least not readily available, technology. However, this is a prototyping platform, and making it harder to work with the platform is counter intuitive. Even though it’s not a sexy material the best all-around way to mount things, at least for prototyping purposes, remain as Velcro. It’s easily adaptable, can be cut and fitted to almost any shape, and the best part is that you can get as much as you want from your local fabric store.
nuSense is based at its core on Arduino, as it’s open source and already has a huge community behind it. It’s a very capable platform, and development is ongoing. While this may be seen as piggybacking on already existing technology it’s by far more effective than developing an entirely new core platform and try to get users into that. Arduino allows users a wider variety and compatibility with existing technology far surpassing that of any newly made platform.
The aesthetics of the modules are left simple and utilitarian to blend in rather than stand out, allowing you to test out your prototypes without having the modules sticking out, or making more of a fashion statement than you need. What’s even better with keeping it all simple is that the more advanced users can easily buy third party sensors and actuator to develop with and make their own enclosures for those using any number of techniques, such as 3d printing or even wood working.
Just having the hardware does not mean it magically knows what to do, you also need a way to access and set up the sensors and actuators to make them do what you want.
The intention behind the nuSense IDE was to leave as much of the complex levels of developing in the background. Instead of writing code on how you want nuSense to behave you access a graphical interface, using sliders, check boxes, easily understood names and values instead of abstracted data. A distance sensor may work on the concept of near to far instead of 0-1023 for instance. While it removes some of the granularity the sensor is capable of it adds for more in ease of use.
Once you’ve gotten your kit, read through the quick start guide and installed the software you’re presented with what is essentially a wizard, taking you through the creation of your first project. First you choose which sensor you want to use (1). Once chosen you set up how it should trigger (2). Next step is to choose the placement on the body (3, 4). With the elastic Velcro that ships with the kit placement becomes entirely up to you. Some areas are recommended, but the software won’t prevent you from placing it elsewhere. The idea being that the frustration of having something not working as intended once you build it is less frustrating, and more of a learning experience, than dealing with software that restricts you from doing what you want.
A similar setup phase is done with the actuator: which one (5), how it should trigger (6) and placement (7, 8). At the end of the wizard you’re presented with what you will need to build the project you’ve designed, and how to plug it all together(9, below). After everything is hooked up you simply connect the base unit via USB to the computer to load the project code onto the device. Wear it and experience the world in new ways!
Beyond the wizard interface lies a mode which you can activate once you’ve reached the boundries of what you can do with the nuSense kit alone, or for that matter if you’re already comfortable with programming and physical prototyping. nuSense is a system designed not only for the novice user, but also for those with a greater wish to push the field of wearable technology further. When you activate the super-user side of the interface you can write your own code for what you want the sensors and actuators to do. Or for that matter you can set up a project using the wizard and then dive into the code of that project and modify it to your heart’s content.
Even though nuSense is set up for ease of use, like with all systems, users will run into issues. While some issues, like broken hardware, would need to be escalated into actual support cases most of what you run into can be solved by other users who have run into the same issues.
The nuSense documentation has plenty of help and troubleshooting, but static documentation only reaches so far. One major element of the nuSense website is the community, where people can go to search for help, to share what they’re working on, to collaborate and just hang out. It’s a source of inspiration when looking for ideas, and from a corporate point of view it’s also free support.
Early during this project an interview was conducted with Paul McCarthy, who together with his son Leon built a prosthetic hand using a 3d printer. He mentioned the importance of co-creation and how their project was fueled by the exchange of ideas and discussion between himself and Leon. One doesn’t have to look further than to the vast amount of communities online solely made for the purpose of exchanging ideas, or to the Maker movement which thrives on this exact kind of exchange.
Eventually, as the community grows and some users end up spending more time and effort there than most, they can be elevated into forum administrators tasked to help keep the forum running. These are also potentially the people who would help to push nuSense forward, to help develop new modules and to suggest important updates and changes to future releases. Some of these “super users” could very well also be made part of the nuSense team to encourage further growth.
As nuSense is based on Arduino technology there are also vast communities already set up to cater to it, so while the nuSense community is solely for the nuSense platform, the ease to get help can be extended far beyond its own community.
It’s fairly easy to see how this platform could be useful for those developing wearable technologies, regardless if that’s designers and engineers working either in consultancies or in-house, or even at schools or those doing research. Currently if you want to experiment in this field you need to find a developing platform and try to bend it into a wearable shape, as no good wearable development platform exist. Sensors and actuators are currently not adapted to work on a wearable level, and experiments tend to become cruder than they need to be. Speeding up the ability to iterate between versions also means that experimentation becomes freer and less costly. Companies could be more inclined to venture beyond armbands and glasses through a greater extent as research and development cycles go faster, and become cheaper.
A good example may be in Google Glass. Looking below the development stages of Google Glass over two years can be seen, and how they started development with hacking a phone, and eventually building up to custom hardware, etc. While there is nothing wrong with hacking existing devices, we’ve all done it at some point, one may still ask if the development cycles couldn’t have been shortened and even some cancelled out by using a development platform already tailored for the purpose?
As nuSense would fundamentally be an open source platform it would also democratize its use, as anyone would have access to it, regardless of prerequisites (aside from having access to a computer). This also means that anyone could develop not just new peripherals and modules for nuSense but could essentially clone the platform.
A good example how this would work could again be seen in Arduino. The Arduino platform is entirely open source, all the way from the code to the circuit board design. Nothing is stopping anyone from opening up their own Arduino manufacturing and selling Arduino compatible boards. This is with one exception, they cannot use the name Arduino as it’s trademarked to Arduino. A similar framework would be set up around nuSense, where anyone could clone nuSense to their hearts content but have to keep the nuSense branding off it, calling it at most nuSense compatible.
This kind of development opens up for a lot of possibilities, just as for Arduino it meant the release of many tailored clones, for different use cases it could mean the same for nuSense. Arduino clones have come out in all shapes and sizes, being able to run almost anything imaginable. What exactly a similar exploration would mean to nuSense is only up for speculation, but could definitely include a much broader set of sensors and actuators, as well as modules for more extreme situations.
As a bonus, while finalizing the project time was spent observing and interacting with a class of second year interaction design master students at Umeå Institute of Design during a project dealing with wearable technology. The hardships they went through both in trying to make quick prototypes to test ideas, as well as building their final concepts was very telling for the need of a better wearable development platform.
The second hurdle was often simply understanding the data received from the sensors used, and how they related to what the user was doing. When you turn a knob on a box, it’s not hard to tie action with received input together, but when things start becoming more arbitrary and you have to take more complex sensor data into account it’s often hard to reach anywhere meaningful, and you do so simply by virtue of spending a lot of time.
nuSense definitely fills a function when being able to iterate quickly on your ideas, as going back and making changes to your project is as easy as hooking up the USB cable and loading over new project code. Even if you need to go into the code and change bits and pieces yourself it’s still a lot easier simply by the code being pre-generated for you and you don’t have to worry about how to communicate with the sensors and actuators, but only focus on the values of how they should act.
As nuSense is grounded in technology of today, while trying to remain a cost effective platform, some concessions had to be made. Cables were used instead of emerging technologies such as Bluetooth Smart (formerly Bluetooth LE) which would allow the modules to be wireless, but would increase the cost, power requirements and size of the platform modules exponentially. If current battery technology continues to improve and wireless communication chips become constantly cheaper, it’s definitely not far-fetched that these modules could become entirely wireless in future versions.
As the current form of nuSense is utilitarian in nature there would also be a lot of work needed to bring it up to a more attractive level. This could though be something for version 2 or 3, when early adopters and the community has had a chance to give input and brought the system to where they see fit. By the time an official nuSense version 2 or 3 is ready for launch more tailored clones for specific situations may already exist on the market. This is not a bad thing as it would leave nuSense to focus on being the main backbone behind the development, rather than designing for every specific use scenario, if one could even map out such a thing.
While it’s doable to draw out guidelines for how to design for wearable technology, things like smooth shapes and soft materials, it’s only after it’s been tested and vetted by users you know if it would work on not. It’s unwise to expect nuSense to bring a revolution to the ergonomics of wearable technology, as that’s not the main point of the system.
Based on [Buxton, B. (2007). Fig 149 & 150 [Illustration]. Sketching User Experiences (p. 388). Elsevier/Morgan Kaufmann.]
Thank you!
Who doesn’t like to share their knowledge?
If you had a talking head, what would you make it say?
RF Communication between Arduinos? Let’s make a shield!
Tweet for a cookie? Sign me up!
