Infinite Interfaces

Introduction

As technology advances and one product can propose a various of different solutions for the user, it became a necessity to design for adapt to user's needs. Traditional interfaces can struggle to keep pace with changing use cases and lack flexibility in adding new features, that could potentially result either in redesign, quick-fix solutions, or division into separate products.

Infinite Interfaces dynamically adapt and change based on user inputs, providing personalised and contextually relevant experiences, bridging the gap between people and technology.

Users seek intuitive interactions, and with infinite interfaces we believe we can revolutionize human-computer interaction and offer a personalised user experience, by offering adaptive experiences, deliver tailored content and functionalities, transforming complex tasks into intuitive processes. This empowers users to effortlessly navigate digital environments and access desired information or features.

Definition of infinite interfaces

In this research, we define Infinite Interfaces as those, that dynamically adapt and change their output and UI based on user inputs, providing personalized and contextually relevant experiences. Such interfaces don't have all its features displayed instantly, but show only the relevant parts of the interface upon user's request.

Similar design approaches

Real-world examples

There are a multiple of examples of Intinite Interfaces currently exhisting, including Wolfram, Siri, Spotlight, Alfred etc.

Some of them are described previously in our previous research article Single Input Interfaces.

What combines them all — is a similar approach to user-computer interaction. The user has a single input field, where he describes what he is rearching for, or a task, and the system creates the output based on it, showing the user only the relevant to him information. The output can also be in different formats — such as web results, text suggestions, calculations, opening 3rd party apps (Siri, Alfred), or even transcribed mathematical equations (Wolfram).

Dynamic vs Infinite interfaces

Dynamic Interface is also a design practice based on a user intent. Basically, the interface changed according to the user's needs, and adapts to the conditions. Choices made by users drive what controls are displayed so that only relevant fields are presented. This approach helps user achieve his goal faster, reduces visual clutter, and shows what's only relevant to the user. In Dynamic interface, user is still presented with an interface

Infinite Interface is an approach that can grow further from the dynamic design practice. Having AI on our hands, we now can not just adapt the interface to user's need, but completely get rid of the interface itself and base the interface on the user's intent only. Having that approach, we believe we can personalise experience for the user, don't overload him with unneeded or unused features, and make the task completion faster, more precise, user-friendly, and simplify the products and interfaces that require domain-knowledge. By reducing the distance between the user and the interface, the interface creates a more direct feel, minimising the user's effort to achieve their goals.

User Interaction and Engagement

User interaction in infinite interfaces

One of the first mentions of a similar approach was in the work "Human-computer interaction" from 1985, by Lawrence Erlbaum Associates, Inc., Edwin L. Hutchins, James D. Hollan, and Donald A. Norman. In this work, they describe the first interfaces, that could be interacted with directly, but not by commands.

The first examples they describe as following:

For example, Bill Budge's Pinball Construction Set (Budge, 1983) permits a user to construct an infinite variety of electronic pinball games by directly manipulating graphical objects that represent the components of the game surface.

Naturally, years has passed and technology have evolved immaculately, but the approach to human-computer interaction in general didn't change as much as we think it did. Now we still have to consider the same things within the user-computer interactions — the user's approach to a task, the cognitive load, language of the interface, speed of the output, and the overall directness of the manipulation.

Shneidermnan (Ben Shneiderman, Behaviour and Information Technology, 1982, vol.1 no.3, 237-256) has suggested that direct manipulation systems have the following virtues:

• Novices can learn basic functionality quickly, usually through a demonstration by a more experienced user.
• Experts can work extremely rapidly to carry out a wide range of tasks, even defining new functions and features.
• Knowledgeable intermittent users can retain operational concepts.
• Error messages are rarely needed.
• Users can see immediately if their actions are furthering their goals, and if not, they can simply change the direction of their activity

And to this day, those criteria fit perfectly to the approach of the Infinite Interface practice that we will be covering in this work.

Two aspects of directness: distance and engagement

The distance in this case depicts the distance between the user's intent and the output of the system. The engagement represents the feeling of manipulating the interface and object of interest directly.

In the "Human-computer interaction" work, the authors introduce terms as "Gulf of execution" and "Gulf of evaluation", depicted in the following picture.

So, in order to create a smooth uses-computer interaction, in the Infinite Interfaces approach, our goal is to shorten or completely erase those gulfs.

The Gulf of execution can be bridged by matching the output of the system to the user's thoughts and goals, and the Gulf of Evaluation — by making the output represent the result in the most clear way, that user can easily perceive.

The bigger are the Gulfs of Execution and Evaluation, the more cognitive load we put on the user in the human-computer interaction, and the goal is to take this load from the user and make it work on the end of the product.

Two forms of distance: semantic and articulatory

Historically, most interfaces have been built on the conversation metaphor.

Previously, in order to achieve his goal, the user had to use the language of the interface. Learn, what the interface means by specific terms, and adapt to using them.

With the AI technology we have now, this conversation can become not a metaphor, but a reality, and this conversation between the user and the system shouldn't feel difficult, technical, and frustrating. This conversation can now be human-centred, using natural language, that will not only help the user, but also make him feel understood, and also in charge of his interaction.

With Infinite Interface approach, our goal is to flip this approach, and make the system understand the user, not vice versa.

In order to shorten the gulf between the user and the system, the output vocabulary of the system should match the input of the user's request. If the output doesn't match the user's language, the cognitive load increases and therefore put the burden of shortening the gulf of distance onto the user.

The more that the user must provide, the greater the distance to be bridged.

One way to achieve articulatory directness at the input side is to provide an interface that allows the user to use more direct sources of input — such as voice commands, or using keyboard to select the options, so that the user's actions and output are perceived more naturally.

The Gulf of Evaluation

When the users observes the output value, he has to evaluate it in terms of how it fits to his request. If the action or the output only shows the current value, the user has to put his mental effort to evaluate and compare the output with the current state and understand whether it changed or not.

In the "Human-computer interaction" work, it's represented by a simple case of filling a water tank. If the interface doesn't show the changes, the burden of evaluating it goes to the user, increases his mental and cognitive load, and make the Gulf of evaluation his responsibility.

By clearly indicating the changes in the interface we can bridge the Evaluation Gulf on the side of the system.

Another example of shortening this gulf is to provide consistency with semantics and actions in the interface, or provide a higher-level language that will include the requested input.

That approach will surely require more effort from the system, but can achieve the shortening of the gap between human-computer interaction.

The learning curve

In the Infinite Interface approach, as well as with all other interfaces we also should consider the user's learning curve. As with all interfaces, the more familiar user becomes with it, the easier and smoother the interaction will become. Meaning, that with time, user will adapt to the system more and more and will understand what he, as a user, is expected to provide to the system.

The full schema of the Execution and Evaluation Gulfs are represented below:

Cognitive load aspect within the infinite interface design framework

At the root of the Infinite Interface approach, we have the assumption that by taking out the unneeded and unused features from the interface, let the user directly manipulate the interface by his input, and only propose the results fitting the user's needs — we will lessen the cognitive load the user experiences.

To address the cognitive load within the interface, firstly we need to understand the Cognitive Load Theory (CLT).

Here is a quote from Fred Paas, Juhani E. Tuovinen, Huib Tabbers & Pascal W. M. Van Gerven (2003) "Cognitive Load Measurement as a Means to Advance Cognitive Load Theory", Educational Psychologist, 38:1, 63-71

CLT is concerned with the design of instructional methods that efficiently use people's limited cognitive processing capacity to apply acquired knowledge and skills to new situations (i.e., transfer). CLT is based on a cognitive architecture that consists of a limited working memory with partly independent processing units for visual and auditory information, which interacts with an unlimited long-term memory. These structures and functions of human cognitive architecture have been used to design a variety of novel efficient instructional methods.

According to CLT, multiple elements of information can be chunked as single elements in cognitive schemas, which can be automated to a large extent. Therefore, the main goal of a CLT-based interface would be creating and automating schemas. However, before information can be stored in schematic form in long-term memory, it must be extracted and manipulated in working memory.

These instructional methods have proven successful for children, young adults (for an overview, see Sweller et al., 1998), and older adults (e.g., Paas, Camp, & Rikers, 2001; Van Gerven, Paas, van Merriënboer, & Schmidt, 2002a) in a wide variety of domains, such as physics, mathematics, statistics, computer programming, electrical engineering, paper folding, understanding empirical reports, and learning to use computer programs.

Compared with conventional instructional tasks, CLT-based tasks have been found to be more efficient because they require less training time and less mental effort to attain the same or better learning and transfer performance.

According to Nielsen Norman Group, when talking about the user interface, cognitive load can be cause by 3 main factors:

• too many choices
• too much thought required
• lack of clarity

In the field of user experience, we use the following definition: the cognitive load imposed by a user interface is the amount of mental resources that is required to operate the system. Informally, you can think of mental resources as "brain power" — more formally, we're talking about slots in working memory.

Intrinsic cognitive load is the effort of absorbing that new information and of keeping track of their own goals.

In the basis of Infinite Interfaces lays the foundation of eliminating the cognitive load for the user as much as possible, which means reducing visual clutter by taking out the features the user doesn't need to use, giving him full control to describe his intent, propose fitting and relevant options to the user's request, and offload the tasks from the user, providing him the actionable results as well.

Short-term memory limitations dictate a whole range of other design guidelines:

• quick response time, so that the user doesn't have to keep in mind what task he was performing
• providing clarity within the interface, to show what task is being performed

At the extreme, only about 5% of the population has enough brainpower to perform complex tasks with computers, such as making high-level inferences using specialized background knowledge. Most likely, you're in this elite group. And, worse yet, so are many other members of your Internet team. (And the rest are definitely in the top 25%, which is also much better than average users.)

Even though it's a bad course title, it's a good overall mnemonic to design for cavemen and their literal-minded and limited-capacity brains. After all, your paying customers are only one step out of the cave.

There are also basic UX Laws, that correspond with the Cognitive Load Theory:

• Hick's Law

The time it takes to make a decision increases with the number and complexity of choices

• Miller's Law

The average person can only keep 7 (plus or minus 2) items in their working memory.

• Jakob's Law

Users spend most of their time on other sites. This means that users prefer your site to work the same way as all the other sites they already know.

In Design Principles for Infinite Interfaces, we will go over how we should approach designing the interface with Cognitive Load Theory and UX Laws in mind.

Testing the interfaces

The task

For this test we conducted a moderated user-testing with 2 different interfaces that have the same functionality: 1 Infinite interface and 1 Dynamic user interface. Both products are system utilities that have to do with computer maintenance.

User's number: 10

Type: Moderated

Task: users were given 3 tasks. The first 2 were straightforward - to find or request a needed feature within the app. The third task was formulated as a problem, that the user faces, so the user needed to understand the root of the problem and complete several steps to resolve it.

Although it is quite difficult to measure cognitive load exactly, we used one of the most simple yet effective rating-scale techniques for this test.

Rating scale techniques are based on the assumption that people are able to introspect on their cognitive processes and to report the amount of mental effort expended. Although self-ratings may appear questionable, it has been demonstrated that people are quite capable of giving a numerical indication of their perceived mental burden (Gopher & Braune, 1984).

Metric used: users' rating on simplicity of completing the task

Results

We noticed that with complex task that require more than 1 action, users rate the simplicity of Infinite Interfaces by 22% better (even if having issues with wording the request), as they only needed to formulate the request in the app, and the actionable items were presented as an outcome already, so the user didn't need to dive into understanding the problem itself, but rather just inform about it and have the proposed solution.

This result was calculated based on the average score users gave to rate the simplicity of completing the given task in both interfaces.

However, even though the task completion rate was much better in infinite interface, the most time user's spent on the task was formulating the request which is connected directly to the Semantic's Gulf explained earlier. In the Design Principles for Infinite Interfaces we go into detail describing how to avoid this issue and make the user-computer interaction as smooth and fast as possible.

Important notes

CLT research has identified various task characteristics, including task format, task complexity, use of multimedia, time pressure, and pacing.

It is also important to note, that learners can compensate for increased mental load, such as higher task complexity, by putting in more mental effort to maintain performance. Hence, the cognitive costs associated with a particular performance level cannot be solely inferred from performance-based measures. Instead, combining measures of mental effort and performance can provide valuable insights into cognitive load.

An interesting observation is that even though researchers are continuously trying to find or develop physiological and secondary task measures of cognitive load, subjective workload measurement techniques using rating scales remain popular, because they are easy to use; do not interfere with primary task performance; are inexpensive; can detect small variations in workload (i.e., sensitivity); are reliable; and provide decent convergent, construct, and discriminate validity (Gimino, 2002; Paas et al., 1994). However, we must stress that the internal consistency of these measures requires further studies.

Determining the effectiveness of instructional conditions requires considering the relationship between cognitive load and performance. Paas and van Merriënboer (1993) introduced a computational method to analyze mental effort and performance to assess instructional efficiency. This approach has been successfully applied in numerous studies. Learners' behavior in specific instructional conditions is deemed more efficient if their performance surpasses expected levels based on their mental effort. Similarly, learners' mental effort is considered lower than expected if their performance exceeds anticipated levels. Learners can compensate for increased mental load by investing more mental effort. However, performance-based measures do not consistently reflect the cognitive costs associated with a particular level of performance.

Design Principles for Infinite Interfaces

• Provide clear instructions in the beginning

The way customers perceive your product is greatly influenced by the information they gather from other products and apps, forming a significant part of their mental models. (Jakob's Law) This type of interface is rather new and unusual for the average user, so in the beginning of the interaction it is critical to minimise the stress and cognitive load as to how to interact with it. Enhance users' mental models by aligning them more accurately with your system. This can be achieved through improved explanations and clearer labels, leading to a more transparent UI. These aspects can be provided during onboarding, as well as through tips and hints at the beginning of the user interaction with the product.

• It's all about the semantics

Taking that Infinite Interface approach is based solely on the user input, this part is the most critical to approach in the right way.

• In the initial phase, help the user understand, what types of input he can do. Give the user and example or a prompt to start with

• Output should semantically match the input. If the user doesn't have enough domain knowledge, the different output can confuse him about whether it is his desired outcome or not.

• Take into account human errors, typos and mistakes. The system should recognise the request even with typos in it, as we don't want the user to spend the majority of his time in the product correcting his mistakes.

• The system should understand the general idea of the input and not solely base it on the words in the input. For example, is the user types "my computer is hot", the system should also recognise requests such as "overheated", "warm", "slowing down", "noisy" etc., and give the same relevant output, resolving the core of the issue.

• Personalise the user's experience

Automate what can be automated. Interfaces have the advantage of having access to contextual data, allowing them to anticipate user's needs rather than relying on the end user to initiate actions.

If the user goes through certain scenarios on a regular basis - propose them to him first, without having to request it every time. You can also base the output on the previous user's actions, or propose next steps for the user in a certain scenario.

• Different types of output should be displayed differently

Let's say, your types of an output could be a text suggestion, a link, a picture, or another app proposed to use. When user sees different types of outputs, he should understand they are different from one another, and after some time with learnability curve, immediately distinguish which output he needs without reading all the options, but just having a glance. It can be done via color difference, icons, sizes, etc.

• Take learnability into account

When you put this design to test, take into account that this type of interfaces is not yet very familiar to the user. So, to understand the real user flow, test the design in the beginning, when user is just getting familiar with it, and also some time later. As the user get's more familiar with system's capabilities, you might see changes in the user's requests and general approach to the tasks, and improve the product based on the new data.

• Don't clutter the space

The idea of Infinite Interfaces is not to propose as many options as we can to the user, but to propose only relevant ones in the specific time and place. So, if the amount of outputs is more that 5-7, the system should filter the most relevant to the user itself, not putting this task on the user. Again, minimising the cognitive load is one of the basis of this approach, so in order to do that, we make the system do the thinking instead of the user.

• Propose options

If the user's input is not clear enough to propose one certain solution, the system should provide a few relevant options to chose from, so that the user doesn't have to rethink his request and do it again. This causes frustration, more cognitive load, and overall poor product satisfaction rate.

User experience considerations

User is put in control of the action While we are trying to minimise the cognitive load with the Infinite Interface approach, we still have to remember that the user is in charge of your products actions. No matter how intuitive a product may be, there will always be moments where the user needs to take control and make a decision or formulate his intent. By keeping this in mind, we can ensure that our designs are both user-friendly and empowering.

Formulating the intention If the user is not introduced to the product and its features and approach properly, there is a chance he might struggle to formulate the intent at first. Having that issue will completely disregard the products output, no matter how good the system is on the background, if the user cannot reach the results he needs. It is critical to adapt to the user's language, not make the user guess how should he form his inputs, and make the input for the user as clear and user-friendly as possible from the very beginning. To avoid that, we can propose user the variety of relevant options, which can minimise the chance of an invalid result.

Potential Future Directions

We envision an interface where a solitary input can bring diverse outputs: text, apps, actions, ushering an era where evolving interfaces reshape engagement paradigms and redefine human-computer interactions.

We see multiple areas where such interfaces can be used:

• Personal Assistants and AI Companions This interface can functions as an advanced personal assistant. Users can express their needs using natural language, and the system can offer text suggestions, launch relevant apps, display images or videos, and even perform intricate tasks like disk space cleanup, all tailored to the user's context and requirements.

• AR/VR The interface in AR and VR environments effortlessly merges the digital and physical realms. Users can voice their desires and receive visual or auditory feedback that overlays their surroundings, providing just the needed information without cluttering the space.

• Internet of Things (IoT) Devices By utilising this interface, IoT devices can enhance usability. Users can interact with their smart home systems simply by expressing their needs, while the system takes care of controlling lights, adjusting thermostats, playing music, and displaying relevant information on smart displays. Users can also use the natural language to ask the system to create a certain atmosphere or a setup.

• Healthcare and Accessibility This interface is transformative for individuals with disabilities. Voice-activated control over a wide range of functionalities promotes inclusivity and accommodation.

• Education and Learning Such interface can be in hand as learning material as well. Users can ask questions related to their studies, and receive visual aids, interactive learning materials, games, and even practice speaking with an AI-assistant within the system.

Conclusion

Infinite Interfaces is a design practice that is logically evolving from the Dymanic interfaces. Having the AI technology we have now, we can now personalise user experience and reduce the time for the user to complete the task and used cognitive effort to do so. Using the described Design Principles for Infinite Interfaces, products can create interfaces that are easy to use and human-friendly. We see a big potential using those interfaces in varioud fields, such as Smart assistants, AR/VR using app, IoT, Healthcare, Accessibility and Educational field.

References

1. Cognitive, physical, sensory, and functional affordances in interaction design. Rex Hartson. Pages 315-338
2. SYSTEMS DESIGN User Interface Design Output Design Input Design Menu Design. Paul Moya
3. Minimize Cognitive Load to Maximize Usability. Kathryn Whitenton on December 22, 2013
4. Short-Term Memory and Web Usability. Jakob Nielsen on December 6, 2009
5. Website Response Times. Jakob Nielsen on June 20, 2010
6. Improving the Design with Dynamic Interface. Test-Driven Development for Embedded C — Pragmatic Programmers (106 / 173)
7. Direct Manipulation Interfaces. Edwin L. Hutchins,James D. Hollan &Donald A. Norman. Pages 311-338
8. System Dynamics Review - Winter 1996 - Vicente - Improving dynamic decision making in complex systems through ecological
9. Cognitive Load Measurement as a Means to Advance Cognitive Load Theory. Fred Paas, Juhani E. Tuovinen, Huib Tabbers & Pascal W. M. Van Gerven
10. BEN SHNEIDERMAN (1982) The future of interactive systems and the emergence of direct manipulation , Behaviour & Information Technology, 1:3, 237-256