For those who don’t know him, John Carmack II is one of the premier game programmers. He was a co-founder of id Software, which you may have heard of. Among his credits as lead programmer are Wolfenstein 3D, Doom, Quake, Rage and their sequels. And he’s the founder and lead engineer of Armadillo Aerospace.
In this article for #AltDevBlogADay, he discusses latency, one of the major problems in VR. Absolutely worth the read.
This will probably end up with a supervillain or something worthy of a Micheal Crichton novel, but it’ll be fun while it lasts…
And the football is pretty cool.
According to Twitter. But hey, we’re better off than those miserable sobs in Louisiana and Mississippi…
The book really shows its age here. I think my favorite part was when a couple of paragraphs were given over to describing how to use a scan converter to grab the monitor outputs, with the appropriate caveats about how (RS-170) video won’t have very good resolution, etc.
The questions about reliability and validity with respect to testing are as valid as ever. The question is whether the methods described in this chapter are really performed as described any more. It seems that testing is being “crowdsourced” these days. For example, Google is known for trying out variations of it’s interface to a subgroup of (randomly chosen?) users to see how user behaviors change using “Mouse Move Heatmaps”, for example. The analysis of the results can be entirely(?) automated, with a defensible numeric set of values being provided to indicate the usability of a website or page.
The combination of massive sensing and analytic capability combined with a very ad-hoc method of software development means that
I would contend that the above conditions are probably both more valid and more reliable than almost anything that can come out of a classic usability lab. This may explain why the only thing that I see regularly being studied in labs are user interfaces that are dramatically different from anything in use. Because if it’s different enough, then there is no user base, and the need to pull out meaningful information from a small sample set becomes critical to the initial development of the technology, or at least in how it is embraced by users.
With that in mind, I would contend that the “Thinking Aloud” method may actually be the best method for obtaining this information. The very “general-ness” and lack of constraint means that the structure of the test is least likely to influence the outcome. Issues can emerge. If these issues emerge consistently enough, then they can be addressed to see if they continue to emerge. Over time though, more data is gathered with the interface, which means that analytics can start to contribute. At some point a handoff occurs, and we’re back to analytics. I’d be curious to see how this process worked with the introduction of tablets and multi-touch interfaces.
First, here’s a spreadsheet that shows all the comments. Nice comments are in blue, problems are in red:
To aggregate, I’ve added a column on the right that has a “meta” description of the issue. After sorting, it can be seen that there are four problem issues that relate to “help”, and then three issues regarding layout and formatting. There is one “Transition” issue.
Clearly,the main issue here is that there is no sitewide help. The other two issues are primarily appearance-related. This isn’t surprising. Content-management systems are pretty good about providing functional websites. Making them uncluttered and is another matter – particularly in volunteer organizations where everything is done by committee These sites tend to have lots of content, but no aesthetic theme.
Reflecting on this exercise, I’m of two minds. First I’m glad I don’t do this for a living. I would go mad. Which leads me to my next thought. Does anyone like doing this? Or is this still done mostly because we have yet to figure out how to do this using automation? I’m not talking about bug-tracking here, because that is more tied into the process of development (for example, some incorrect calculation feature in a spreadsheet). And in fact, bug-tracking and fixing is reasonably mature. We have white box and back box tests that are run under continuous integration systems. And yet there seems to be nothing of the kind for GUIs.
Recently, there has been some research in the use of Machine vision for this kind of testing: [http://groups.csail.mit.edu/uid/projects/sikuli/sikuli-chi2010.pdf], [http://link.springer.com/article/10.1007%2Fs11219-011-9135-x?LI=true]. This work seems to point to the possibility that the era of tedious, manual examination of GUIs may be coming to an end. Personally, I can’t wait.
Well, it does look like there is a product – Eggplant, which is a vision-based system, according to Wikipedia. And here is someone’s list of tools.
Google wants to hear from you about coming up with stuff for Glass…
An interesting article in the Washington Post on athletes gathering data on themselves. As an owner of Polars, Powertaps, Garmins, an iBike and too many bike computers to count, I’d say that from my perspective, we’ll put up with almost any UI, as long as. The. Device. Always. Works. Lost data is incredibly frustrating.
[All quotes are from Jakob Nielsen. Usability Engineering. Kindle Edition.]
An interface should not look like an angry fruit salad of wildly contrasting, highly saturated colors. I swear, the story of my career could be trying to make otherwise intelligent developers understand this….
Even better, the initial use of the training wheels interface did not impair users when they later graduated to the full system. On the contrary, users who had learned the basics of the system with the training wheels interface learned advanced features faster than users who had been using the full system all the time. It’s a nice thought, but I’ve never seen a training wheels version of code pretty much ever. The cost to do so must be prohibitive.
let the users vote on the names, based on a shortlist of possible names. This list can be generated by several means, including suggestions from the developers, from usability specialists, and from asking a few users. This is a great idea that I had never heard of. I’m going to try it in the future.
They should be precise rather than vague or general. For example, instead of saying, “Cannot open this document,” the computer should say “Cannot open `Chapter 5′ because the application is not on the disk”. At this point in web ubiquity, I’d have scraped forum posts as potential sources of help. It’s rare that the developers can come up with all the contingencies, but the users run itno problems and then post about it.
In one experiment, adding different sound effects to each of the modes in a computer game decreased the users’ mode errors by 70%. Mode feedback, particularly in some of the more sophisticated programs (Photoshop, I’m looking at you), could really benefit from good sound design.
Two further differences between heuristic evaluation sessions and traditional user testing are the willingness of the observer to answer questions from the evaluators during the session and the extent to which the evaluators can be provided with hints on using the interface. I’ve done this a lot. It’s fast, effective, and gives a really good idea what the quickstart (text or video) should have in it.
Another way of utilizing different kinds of expertise is the pluralistic usability walkthrough technique [Bias 1991], where the heuristic evaluation is performed by representative users, product developers, and usability specialists. In some ways, this can be the most effective way of training trainers. You get all kinds of benefits: your power users start to get buy-in for the system, since they are now participating in the development of it, and the users who are going through this process give the developers the perspective of a user that is progressing from naive to sophisticated. It’s really the only way to get this sort of insight, since as stated earlier in the book, “It is impossible to unlearn an interface”
User Analyses (General)
Pew Internet has some good information as well:
Task Analysis
HTA
Environment Analysis
The site (or more accurately the content of the site should be accessible from a variety of locations. Users at teh home should be able to reach the site from their home computers and tablets. Mobile users should be able to reach the site from their phones Potentially other means could be used to reach users. For example, text messages could be sent to subscribers who want to be informed of newly posted rides in a particular category (weekday rides, for example). Email listserves could also be exploited for similar purposes. The idea is to have the information from the site be visible anywhere the users might find convenient. For example, pictures from bike tours could be cross posted to Pinterest. Upcoming rides could be shown on Facebook. Mashups of data (traffic, weather, GIS) could be combined to allow a rider to set an alarm so that they have time to get dressed, load the bike and drive to a ride start.
Profiles (created with help from from http://www.fakenamegenerator.com):
Benjamin M. Hawkins
Description:Ben is a core member of the Baltimore Bike Club (BBC). He has been a cyclist since he turned to it as a way to lose weight in 1996. Since then he has raced some, gone on several bike vacations, and has a collection of bikes that makes his wife question his sanity. He does enjoy showing up for rides in his Fiat, that he picked up on one of his cycling trips in Europe. One of his main uses of the internet is to support his hobby. He prowls Ebay for deals. He posts his data to Strava and Training Peaks. Aside from family, most of his Facebook friends are other cyclists. He knows his way around the BBC website reasonably well. He uses the calendar to find and occasionally lead a ride and visits a forum now and then. He occasionally check out other club information like ride starts. He finds the website ugly but workable, and hates it when it changes, since he has to learn where everything is again.
Jeanette L. Griggs
There is a company, sixense, that looks to be making some very good immersive hardware and a free(?) api. And they have some nice sound (midi) code too. It’s somehow tied up with Intel’s perceptual computing effort. I learned about this from a slashdotted article about the Holodeck Project.
Hye-Kyung’s HCC729 Blog keiths729 Unknown Feed Shelley’s HCC729 Blog Unknown Feed HCC 729 Unknown Feed Galina|HCC 7.2.9 Unknown Feed Judith’s HCC 729 hcc 729 syhcchcc729 Some Assembly Required Marie in HCC 729 hccmaster Mr. Dai’s HCC 729 Blog
PhccSevenTwentyNine 