Amazon CodeCatalyst Logs

Crafting an effective observability Log Management System

In 2023, AWS unveiled CodeCatalyst, a cloud-based tool designed for seamless collaboration among software development teams. This platform offers a centralized space for planning, code collaboration, and the end-to-end application development process, integrating continuous integration/continuous delivery (CI/CD) tools. During this time, I contributed to a project aimed at crafting the the best log management experience to enhance observability.

Quick Glimpse

Crafting Impactful User Experiences

Project Overview

CodeCatalyst is a cloud-based collaboration space for software development teams. It provides one place where users can plan work, collaborate on code, and build, test, and deploy applications with continuous integration/continuous delivery (CI/CD) tools. The log management feature records the event history of account activities. It provides administrators visibility into how resources are being utilized, altered, and modified inside organizations and projects.

My Role and Team Composition

Metrics and KPIs

In 2022, as a UX Researcher and Designer, I participated in the project focusing on enhancing the log management experience for a new SaaS offering within AWS's CodeCatalyst framework. Collaborating closely with stakeholders including the 1 Project Manager, Engineering Team, 1 Software Architect and fellow UX Researchers & Designers, I utilized a suite of tools such as Figma, Balsamiq, Quip, and UserTesting to drive the project forward.

Logging Adoption Rate
Event Monitoring Frequency
Error Detection and Resolution Time
User Engagement with Logged Events
Logging System Performance
Accuracy of Logged Events
User Satisfaction with Logging Interface
Impact on Security Incident Response
Compliance and Audit Preparedness
Cost Reduction in Troubleshooting

Business Objectives

UX Design Objectives

Enhance Security and Operational Efficiency
Enable Auditing and Monitoring
Support Troubleshooting
Facilitate Permissions Management
Identify Suspicious Activities
Promote Transparency and Accountability

Seamless Workflow Integration
Clarity and Intuitiveness
Quick Identification and Context Understanding
Effective Troubleshooting
Role-Based Access and Views
Efficient Event Monitoring and Management

Puzzle Pieces

Navigating Design Challenges

User Insights and Behavioral Analysis

Our customer base comprises small teams that value autonomy and agility. Unlike larger enterprises, these teams do not typically have stringent governance or data retention mandates. However, they still need reliable tools to co-manage security and operational issues promptly. Logs play a crucial role here, providing visibility into activities and enabling quick responses to security and operational challenges.

Based on secondary research, a 2020 field survey of a similar tool highlighted the diverse use cases for log management among users: audit and compliance, security analysis, operational troubleshooting, visibility into activities, and detecting anomalous behavior. These insights reinforced the importance of developing a flexible, user-friendly logging system that could cater to these varied needs.

The Challenge

The core UX challenge was to design a log management system that:

Empowers small, autonomous teams - to quickly and effectively handle their security and operational needs.
Maintains simplicity and ease of use - while providing the necessary depth and functionality to support activities like audit and compliance, security analysis, and troubleshooting.
Balances user-centric design - with technical feasibility, ensuring that the logging system is both robust and accessible.
Differentiates our solution - by integrating best practices and standout features identified in our competitive analysis, making our logging system not only functional but also a preferred choice for users.

Insight Expedition

Mapping the User Landscape

Leveraging Competitive Insights

To shape our logging system design and capitalize on our competitive advantage, I undertook an extensive analysis of 12 existing log management tools. This competitive study aimed to uncover how other services monitor and manage logs and how users visualize and utilize log data. Each tool provided valuable insights into essential functions and standout features that users found particularly beneficial. We aimed to incorporate these insights into our design to create a superior user experience.

Key Objectives of the Research

Identify the log management tools customers currently use.
Understand the workflows associated with log management.
Explore how users engage with these tools in their daily routines.
Identify customers’ needs and expectations from log management systems.
Uncover opportunities for improvement and innovation in log management.
Clarify roles and specific requirements for monitoring logs effectively.

User Research

To deepen our understanding of user needs and workflows, I conducted five remote semi-structured interviews with professionals who regularly interact with log management tools. Our participants included software engineers, data analysts, and DevOps engineers from diverse sectors such as finance, insurance, healthcare, telecom, retail, and port management. These interviews provided us with rich, qualitative data on how these professionals use logs for various purposes:

Monitoring and Troubleshooting: Ensuring operational stability and addressing issues as they arise.
Security Safeguarding: Protecting systems from security threats through vigilant log monitoring.
Infrastructure Migration: Supporting the transition of infrastructure with effective log management.
Software Design and Development: Utilizing logs to guide and refine software development processes.

What Did We Learn from the Research?

Through the research, several key insights emerged that shaped my approach to designing the logging system:

Zero Setup Process: Users prefer a clear and quick setup process. Any complexity or lengthy setup can be a significant barrier to adoption.
One-Stop Visualization: Users favor having a comprehensive view of logs within the console interface. If the provided tools lack sufficient insights, they often turn to third-party solutions like Adobe Analytics, Tableau, PowerBI, and CloudWatch to enhance log visualization.

Fast Search: Users typically search for specific event logs in a reactive manner, meaning speed and efficiency are crucial for their tasks.
Multi-Attribute Filtering: Users need the ability to filter search results by various attributes such as events, time, actions, and resource types. They often use complex queries, leveraging SQL and GraphQL, to build effective filters and find precise information.

Persona

Innovation Hub

Cultivating Design Concepts

Simplifying the Event Schema

In our journey to create a user-friendly logging system, one of the key challenges was to make sense of the Event Schema. The Event Schema is a crucial component that defines the elements and types identifying various attributes of an event. After the engineering team established a comprehensive schema, it was my task to simplify and streamline it for our users, ensuring they could easily understand and use the information.

To simplify the logging system's Event Schema, I focused on defining and refining key fields to enhance user understanding and decision-making.

Critical Fields Selection: I identified nine essential fields crucial for providing clear insights into each event.
Additional Fields: Added two complementary fields to give users a more complete view.
Streamlined Focus: Reduced the schema to eleven key fields, helping users quickly grasp important attributes without being overwhelmed.
User-Centric Design: Guided by user research, the selected fields align with common needs and workflows for easier interpretation.

Outcome:

Faster Decision-Making: Users can swiftly assess event attributes, leading to quicker and more efficient responses.
Enhanced Usability: The simplified schema makes the interface more user-friendly and easier to navigate.
Increased User Empowerment: Clear, concise event views empower users to manage security and operational concerns with confidence.

Wireframing Stage

Refining the Logging System Interface

During the wireframing stage, I focused on iterating and refining the initial design of our logging system interface based on valuable feedback. Here’s how I adapted our wireframes to better align with user needs and our internal design guidelines:

Feedback Incorporated

Split View Design:

We transitioned from an expandable tabular design to a split view layout. This change not only aligned with our internal design system library but also improved the overall user experience by offering a more structured and visually consistent interface.

Multi-Select Detailed View:

To enhance usability, we integrated a multi-select detailed view. This feature allows users to select multiple items and view detailed information simultaneously, streamlining the process of managing and analyzing logs.

Standard Pagination:

We reverted to a standard pagination design positioned at the top right of the interface. This decision was made to ensure consistency across the product, providing a familiar navigation experience for users.

Property Filters:

We implemented property filters from our design system library that closely match user requirements. These filters allow users to refine their searches based on specific attributes, making it easier to find relevant log data.

Primary Navigation Entry Point:

We established a clear primary navigation entry point for accessing the logging system. This focal point simplifies the user’s journey into the log management area, enhancing accessibility and ease of use.

Feedback Not Incorporated (Due to Scope Limitations)

Global Search Integration:

While a global search feature was suggested, we did not incorporate it in this iteration due to scope limitations. Implementing such a feature requires a broader integration effort, which we plan to consider for future updates.

Alternative Entry Points:

Suggestions to include additional entry points, such as from a project viewer, were not adopted in this phase. We decided to maintain a singular, clear entry point to keep the navigation straightforward and focused.

Sub-Navigation for Logs:

The idea to position logs under the organization setting as a sub-navigation was not pursued. This approach could complicate the navigation structure, which we aimed to keep simple and intuitive.

Table Column Swapping:

The feature allowing users to swap table columns was left out due to scope constraints. This advanced functionality could be revisited in future iterations as we gather more user feedback and prioritize enhancements.

Hover Detail Viewer Signifier:

Adding a signifier for the hover detail viewer was suggested but not included in this round. We opted to prioritize other design elements that more directly impact the overall user experience

Pixel Perfection

The Journey of Design and Development

Hi-Fidelity Prototype

Our team had an exciting task of being the first to use aws’s new design system. I designed various features using this deign system, carefully testing how well the new components worked in our applications. While doing this, with my team we noticed areas where the components could be improved and shared this feedback with the agency to help make the system better. Additionally, my team and I held weekly meetings to discuss and improve common design patterns, ensuring the system worked smoothly and effectively for all our projects.

Property Filter

Allows developers to filter logs using a variety of parameters and their combinations.

Date Range

Allows developers to filter dates using either relative or absolute criteria.

Event-Viewer Preference

Enables developers to manage pagination, set the time format, and choose which fields are visible in the table.

Split View

Allows developers to view detailed logs in a split screen. Additionally, they can switch between vertical or horizontal layouts, copy event logs, highlight high-priority fields, and compare multiple events side by side.

Download

Allows developers to download event logs in either CSV or JSON format.

Iterative Innovations

Enhancing UX Through Testing

Usability Testing

The insights I gained from the design review and rapid usability testing sessions were thoroughly integrated into the project. These adjustments ensured that the final design was more intuitive and aligned with user expectations.

Before

After

"It kinda feels like there's an extra step to get to the log data, like you have to go through this unnecessary layer first. It seems a bit overcomplicated."

Anonymous User

Before

After

"I was thinking about using a bottom panel instead of the one on the side. "

Anonymous User

Before

After

"Thought about adding an 'expand all' and 'multi-select' feature. "

Anonymous User

Learning

Working on this project provided me with a profound understanding of observability. The features we developed enable teams to swiftly respond to potential issues, uphold resource security, and much more. With our new system, admin users can now monitor activities closely and take appropriate actions whenever an event deviates from company policy.

Previously, software development teams using our console lacked adequate visibility into their resources. This deficiency often led to delayed responses and compromised the security and efficiency of their operations, ultimately impacting their ability to deliver quality products.

Throughout this project, we not only implemented essential features but also gained valuable insights through research. These insights have pointed us towards future enhancements, such as custom alerts, pattern recognition, and advanced analytical capabilities. In addition, we plan to introduce dashboards that offer more detailed insights and analyses. These dashboards will provide actionable triggers, helping teams to make informed decisions quickly. This will further enhance the system's ability to support teams in maintaining high standards of security and operational efficiency.