This application is designed for 1 human being. That human being is Nicolette.
How can Nicolette copy and paste without a mouse?
Nicolette has very limited ability to use her arms and legs. She does not use a mouse and barely uses the keyboard. So she cannot use shortcuts and the right mouse. Alternatively, she uses a wacom drawing tablet with a pen as a mouse and a voice program as a keyboard. Since Nicolette doesn't use a mouse or shortcut, it becomes more difficult to copy and paste text. That is why creating a prototype where she can copy text without shortcuts or the right mouse button is very important.
As a solution, is there a article for demostration perposes.
By using the speech tool she can say copy and paste and the tool will do this for her.
Instead of shortcuts like ctrl+c she can copy by pressing c to copy the text. And if she presses v the text can paste somewhere.
The concept that was created is a site where the user can select, copy and paste using speech.
git clone https://github.com/RainbowJM/human-centered-design-2223.git
For the installation of this application you will need the following:
- NPM
- NodeJS
- Express
- EJS
- Nodemon
To install NPM, use
npm install
To install NodeJS on MacOS you can use the following command in your terminal
brew install node
After use the following command to set up you package.json
npm init
After doing this you will have to answer a few question so the set up, at the end you will get the following package.json file
{
"name": "human-centered-design-2223",
"version": "1.0.0",
"description": "The copy and paste machine 2.0",
"main": "app.js",
"scripts": {
"start": "node app.js",
"dev": "nodemon app.js",
"test": "echo \"Error: no test specified\" && exit 1"
},
"repository": {
"type": "git",
"url": "git+https://github.com/RainbowJM/human-centered-design-2223.git"
},
"keywords": [
"copy",
"paste",
"table",
"voice"
],
"author": "Jevona Magdalena",
"license": "MIT",
"bugs": {
"url": "https://github.com/RainbowJM/human-centered-design-2223/issues"
},
"homepage": "https://github.com/RainbowJM/human-centered-design-2223#readme",
"dependencies": {
"ejs": "^3.1.9",
"express": "^4.18.2"
},
"devDependencies": {
"nodemon": "^2.0.22"
}
}
For the server side of the application, we used the library Express. To install it you can use npm
npm install express
All the logic of the server you will find in the file app.js.
For this application the template engine that is used is EJS. It will take care of the rendering of the static pages
To install the template you will have to do the following
npm install ejs
In the server you have to tell it where what template is used
This you do by adding the following in the app.js, the method app.set()
And for the static file like the stylesheet, img you have to use de static express middleware
Als you have to indicate that this is in the public file
app.set('view engine', 'ejs');
app.set('views', 'views')
app.use(express.static('public'));This package is life saver it makes it more easier, so that you don't have to restart your server each time you add/edit something in your code. It will take care of this for you.
By using the script options in the package.json you can make different script that you can then run by using a shorter command in your terminal
"scripts": {
"start": "nodemon app.js",
"dev": "node app.js"
},To start the server use
npm start
The structure of the server you will find in the file app.js
const express = require('express');
const app = express();
const port = process.env.PORT || 1000;To import 'Express' you have to use 'require()' method
The localhost port in this case is 3000, this is where the server will be running.
For all of this you have to have your application knowing where it should be listening to.
This part you can do by adding the following code in your app.js
app.listen(port, () => {
console.log(`Example app listening on port ${port}`)
})Description
Study situation is about studying the situation of one specific person, such as that person's physical capabilities and obstacles. By studying the situation, you will achieved the needed improvement in the design, among other things.- Minimum number of clicks required. Or non at all.
Description
Prioritize identity is about personalizing the product for one specific person, by listening carefully to the person in question and getting to know them better. By prioritizing identity, I achieved the design improvements below, among others.- Application is in Dutch
- Used light mode instead of dark mode.
- Color palette is based on the color of nature. Nature is something that Nicolette is fascinated about.
Description
Ignore conventions is about letting go of theoretical design rules, and prioritizing a design that works best for one specific person. By ignoring conventions, I achieved the design improvements below, among others.- Used speech recognition for copy and paste.
- Created simple commands for copy and paste.
- Font size bigger.
Description
Add nonsense is about adding useless features or simply because it's fun. By adding nonsense, I achieved the below improvement in the design, among other things.- When you press the wrong key something will happen in the website.
Nicolette is a 76-year-old woman. Nicolette has limited use of her limbs, she can partially use her arm and legs. She uses a wheelchair to get around. She is trained geologist, and really love topics about climate change and nature. She worked at municipality of Amsterdam, where she organized many activities for disabled people.
Nicolette has a laptop where she stores a lot of documents. To operate her laptop she uses a drawing tablet Wacom Intuos S and Dragon a speech software. She uses her drawing tablet as a mouse where she can click on elements. Because she has more control when she uses the pen. Furthermore, she uses the speech software as an alternative to the keyboard. She can type but not as fast, can press a key one by one. So Nicolette can operate her laptop on her own, but she would like to find an easier way to want to select, copy and paste important piece of text.
Nicolette likes to read articles on her laptop, she uses a wacom tablet as a mouse and the voice software as a keyboard. During testing, you could see that she can use the mouse pad and keyboard a bit. But she does have difficulty selecting text with the pen and mouse. She cannot use keyboard shortcuts such as ctrl + c and ctrl + v. But she did try. This will be accomplished by using the web speech API, so that the user (Nicolette) can copy and paste. Next to that there is also that the user can copy and paste by using simple commands.
src: https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API/Using_the_Web_Speech_API
With this concept it would be easier for the user (Nicolette), but also for a lot of other people, to select/copy/paste without having to use key combinations and precize movement with the mouse.
src: https://www.getfeedback.com/resources/ux/how-user-scenarios-help-to-improve-your-ux/
After the introduction round, was the round were Nicolette could try the first version of the prototype.
Version one of the prototype you can find it in the following branch: Version 1
The this prototype was made based on the informa=tion that we got ans assumptions that I made about Nicolette. The first version of the prototype was a simple HTMl page with nonsense text. For the copy and paste part, You had to select the text by dragging and by using the web speech api I should copy it and paste. But sadly while testing we found a bug, that is why tha prototype wasn't working as it should.
Feedback:
- She found it very difficult because she had to select everything
- She would like to select without scrolling
- She has on her pen a right mouse button on so she can select a little bit, but it she wants to do it with speech
- Make settings of Speech easier to select text.
- She want a easier way to select text.
Plans/Idea's for next prototype:
- Ensuring that you can perform action with one command, key, tap, click.
- Copying without scrolling longer.
- Implement only light mode.
- There should have a clear contrast between the foreground and the background edge.
- Text should be large enough so she can read it.
- Selecting without dragging, instead use voice.
- Scroll down with a key/speech, or move to the side.
- Command that does a few actions, to cut and paste.
- Easier way to turn speech software on and off.
Full insight how the test went step by step
The first test went as follow.She came in you could see already how she manage to transport herself in the wheelchair. She was controlling the wheel chair by moving her feets. She usually uses her second finger to operate her cell phone. That was the fierst observation that I had.
We had a conversation with her, to get to know her better. Here is where she explained to us that the connection between her brain and limbs, doesn't work as it should She can't feel her limbs, or when she touch something. Her central nervous system works fine, but the other nervous systems doesn't. She has spasms that are very painful. She can move her fingers but not all of them. It is a hassle for her to copy and paste.
Who is Nicolette:
- She is a geologist
- She worked at the municipality of Amsterdam
- SHe is president of the disabled organization southeastwest in 2001
- She is 76 years old
- She goes to the gym, since 30 years she goes there, and has a feeling that she can operate more with her fingers. This also allows her to move her limbs better now. Before, she had to ask to lift her legs.
- She could not exercise according to the sports rehabilitation center when she was 40 years old. So she went to the doctor, because she did feel that she cannot live without moving.
- She goes to vacation with her new laptop, drawing tablet and microphone for speech.
- Because she is a geologist she is going to vacation to Iceland, she is going by helicopter. She goes to see spitsbergen.
- She reads mostly on paper, because she's old-fashioned. I think a lot of books.
- She uses all kinds of websites, especially architecture websites, climate change, biology, geology.
- She also reads newspapers.
- Subjects that interest her are, nature, climate change.
- Been very politically active, and she thinks it's not possible if you don't have a computer.
- Mountaineer was her hobby
Nicolette's Devices:
- Acer laptop windows 11
- Uses speech software, Dragon 6
- Drawing tablet, Wacom intuos S, only as a mouse
Nicolette writes a lot, she has a lot of files. It is getting harder and harder to copy pieces from one place and place them in another file folder. The earlier version of windows it was easy to cut and paste. Windows 11 is a little more difficult.
Step by step how Nicolette copy and paste text/files:
- Piece is stored in a document in downloads. And you think that should be in another file.
- Go up to get to the pin
- Then open the file folder and if there are many file folder she has to scroll to go to the right file folder, and she finds it by scrolling.
- She check the file she needs to copy and click the copy icon.
- She has something installed that puts checkboxes at the front of all files.
- When she checks the checkbox, the file is selected.
- Then she goes to scissors, and hope she can click to scissors.
- Then she has to go to the other folder to document (downloads), to share the document somewhere in the other file.
- When she has found the folder she can paste.
Nicolette finds it complicated to navigate with the speech software, she wants to do as much as possible with the tablet that she uses as a mouse. But typing she does mostly with speech on her phone, she showed us how she sends a message to Eric with speech. She can type, because Eric name was written wrong and tap with to press the voice button. She can also hold her phone and say her message. She tapped the send button with her second finger. She finds it a little difficult to scroll( swiping) with her fingers. She did a circular motion, before she presses the send button. It kind of aiming, if her hand does not do immediately, then she try to look from although her hand wants to take the target. Because it's going to take some time. And sometimes it doesn't work because when she has spasm she can't commuicate because it's very painful.
She was asked if she can work in public with her speech tool, but she does not find it comfortable to sit in a cafe with her laptop and tablet and start speaking long texts. Because then everyone will go crazy she thinks.
Nicolette can't double-click, because it's a quick move. So for example if you want to select a piece of text. You press down on the tablet and drag over the text. Then the text turns blue. That works for many people, but not for Nicolette. She can go up and down with the pen but not accurately. Only it doesn't go as fast. Holding and dragging is not yet possible.
src:
- https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API
- https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecognition
After the first testing of the prototype. I continued to work on the prototype, and added more feature to it.
For the second testing op the prototype, I focus on how to copy paste with voice.
Version two of the prototype you can find it in the following branch: Version 2
For this testing I made a test plan. The test plan was the following:
Create a scenario:
- You would like to be able to select important texts, copy and paste it.
Task:
- You have read the text and would like to select, copy and paste it.
She did take a bit to understand, how you have to do this. Mainly because the application was in english. I did have to give a few hints to her. The dragging part is still a difficult part for her. The part where she could use the speech tool to copy and past she liked it.
- Turn on/off the speech tool.
Again because de application was in english she did not understand, the instructions a first. but after giving some hints, she did it.
- Give feedback about the styling in light mode
She like it that it was in light mode, because dark mode made it difficult for her to read
Plans/Idea's for next prototype:
- Change the language of the application
- Copying without scrolling longer.
- Text should be large enough so she can read it.
- Selecting without dragging, instead use voice.
- Scroll down with a key/speech, or move to the side.
- Command that does a few actions, to cut and paste.
src:
After the second testing, and made some fixes and added some more functionality to the prototype. We can call it the final prototype for now.
The final prototype you can find it in the following branch: Final Prototype
In this iteration the part of copy paste based on using simple commands was added.
Also the language of the application was changed to dutch, so it would be easier for Nicolette to understand the instructions.
The final improvement of the prototype was the styling of the application and added the part that when you press the wrong button, the application will give you a sort of feedback.
All the features that was implemented in the prototype you can find it in the following section: Feature.
- User can select one word or multiple words from the text copy and paste it by using speech.
- User can copy the selected text using speech.
- User can paste the copied text using speech.
- Implement only light mode.
- There should have a clear contrast between the foreground and the background edge.
- Easier way to turn speech software on and off.
- Command that does a few actions, to cut and paste.
- Ensure that user can perform action with one command, key, tap, click.
- Text should be large enough so she can read it.
- Add nonsense
- [] User can select one word or multiple words from the text using speech.
- [] User can copy without scrolling long.
- [] Selecting without dragging, instead use voice.
- [] Scroll down with a key/speech, or move to the side.
The prototype is a good start, but there is still a lot of work to do. After testing the prototype with Nicolette two times, each time you understand better what and where her struggle is. One of her biggest struggle is to copy and paste between documentation. The final prototype does not fully solve this problem, because it is a web application. But It does cover the part of copy and paste easier. And you can now copy something from the web application with the simple command or speech. And then go to the documentation and paste it there, by using right mouse click and paste. For the part of copy and paste between documentation, it would be better to create a desktop application.