Posts on Website of Unmesh Gundecha

Architecture Decision Records (ADRs)

Wed, 10 Jul 2024 00:00:00 +0000

Architecture Decision Records (ADRs) are a way to capture important architectural decisions made along with their context and consequences. They are essential for maintaining a clear historical record of why certain architectural paths were chosen over others by the team. This helps in understanding the evolution of the system, facilitating knowledge transfer, and ensuring that future decisions are informed by past experiences.

Structure of an ADR

A typical ADR includes the following sections:

Title: A descriptive title.
Context: Background information explaining why the decision is needed.
Decision: The decision itself.
Status: The current status (e.g., proposed, accepted, deprecated).
Consequences: The expected outcomes, both positive and negative, of the decision.

Architecture Decision Records (ADRs) are typically maintained as markdown documents within code repositories. This practice ensures that the rationale behind key architectural decisions is easily accessible to the entire development team, directly within the context of the project’s codebase. By keeping ADRs in the same repository as the source code, teams can benefit from version control, collaborative editing, and the ability to link decisions directly to the code they affect. This seamless integration helps in maintaining a clear historical record of why certain decisions were made, facilitating better communication, and supporting informed decision-making throughout the project’s lifecycle.

Let’s dive into an example to understand this better.

Example of an ADR:

GitHub Self-Hosted Runner Security Architecture.

Secure Deployment of GitHub Self-Hosted Runners

Context

Our organization is increasingly relying on continuous integration and continuous deployment (CI/CD) pipelines to automate software development processes. GitHub Actions is the chosen CI/CD platform due to its seamless integration with our existing GitHub repositories. However, using GitHub-hosted runners poses certain limitations and security risks, particularly in handling sensitive data and running workflows with elevated permissions.

To mitigate these risks and gain more control over the environment, we have decided to use self-hosted runners. This ADR outlines the security architecture for deploying self-hosted runners to ensure they operate securely within our infrastructure.

Decision

We will implement the following security measures for our GitHub self-hosted runners:

Isolated Environment: Self-hosted runners will be deployed in isolated virtual machines (VMs) within a dedicated Virtual Private Cloud (VPC) to prevent any unauthorized access.
Access Control: Use of role-based access control (RBAC) to ensure that only authorized personnel can manage and access the self-hosted runners.
Network Security: Implementation of strict network security groups and firewall rules to control inbound and outbound traffic to and from the runners.
Secrets Management: Utilization of a centralized secrets management system to handle sensitive data, ensuring that secrets are never exposed in the runner’s environment.
Auto-Scaling and Rotation: Automatic scaling of runners based on workload demands and periodic rotation of VMs to minimize the risk of long-term exposure.
Monitoring and Logging: Continuous monitoring and logging of runner activities to detect and respond to any suspicious behavior or security incidents promptly.
Patch Management: Regular updates and patches to the runner operating systems and software to address any known vulnerabilities.

Status

Accepted

Consequences

Positive

Enhanced Security: The isolated environment and strict access controls significantly reduce the risk of unauthorized access and data breaches.
Controlled Environment: Centralized management of secrets and network rules ensure a consistent and secure runner configuration.
Scalability: Auto-scaling and rotation of runners enhance reliability and reduce the risk of long-term security exposures.
Visibility: Continuous monitoring and logging provide visibility into runner operations, aiding in quick detection and response to security incidents.

Negative

Increased Complexity: The setup and maintenance of the isolated environment, access controls, and network security require additional effort and resources.
Cost: Deploying and managing self-hosted runners in isolated VMs within a dedicated VPC may incur higher costs compared to using GitHub-hosted runners.
Maintenance Overhead: Regular updates, patch management, and monitoring add to the operational overhead.

Example of Architecture Decision Record Repository

Lullabot: Here is an example of how Lullabot manages their ADRs, which you can explore by visiting their architecture site: Lullabot Architecture.
UK Government Agencies: This is another notable example from a UK Government agency capturing key architecture decisions.

Ending

Documenting architectural decisions through ADRs provides a structured way to capture the rationale behind critical decisions, ensuring clarity and consistency in the development process. The example of securing GitHub self-hosted runners illustrates how a well-thought-out ADR can guide the implementation of robust security measures, balancing the benefits against the associated costs and complexities.

By maintaining a comprehensive record of such decisions, development teams can facilitate better communication, improve decision-making processes, and ensure that future developments build upon a solid foundation of informed choices.

Fixing Cropped Mermaid Diagrams in Obsidian

Tue, 18 Jun 2024 00:00:00 +0000

Obsidian is a powerful note taking and knowledge management tool that supports various types of content, including diagrams created with Mermaid.

However, when using the Mermaid syntax you might encounter an issue where Mermaid diagrams appear cropped, cutting off essential parts of the diagrams. This post will guide you through the steps to resolve this issue and ensure your diagrams display correctly.

Understanding the Problem

Mermaid diagrams in Obsidian can sometimes be rendered incorrectly, resulting in parts of the diagram being cut off. This usually happens due to the default settings not allocating enough space for the diagram’s content as seen in below screenshot:

While plugins are available to help fix this issue, I was looking for a quick solution. The first step involves modifying the CSS styles to ensure the diagrams are allocated enough space. Here’s how you can do it:

Step 1 - Open Obsidian Settings

Click on the gear icon in the lower-left corner to open the settings.

Step 2 - Go to Appearance

In the settings menu, navigate to the Appearance tab.

Step 3 - Open CSS Snippets

Scroll down to the CSS snippets section and click on Open snippets folder icon.

Step 4 - Create a New CSS File

In the snippets folder, create a new CSS file. Name it something like mermaid.css.

Step 5 - Add the Following CSS

Open the newly created CSS file in a text editor and add the following CSS code:

.mermaid svg {
    max-width: 100%;
    height: auto;
}

Step 6 - Apply the CSS Snippet

Save the CSS file and go back to Obsidian. In the CSS snippets section, you should see your new snippet. Toggle it on to apply the changes.

The Mermaid diagram now well fits the width as seen in below screenshot:

Ending

By adjusting the CSS styles and modifying your Mermaid code blocks, you can resolve the issue of cropped diagrams in Obsidian. Additionally, using specialized plugins can offer more control and better rendering options. With these steps, your Mermaid diagrams should display correctly, ensuring that all parts of your diagrams are visible and properly rendered.

Sankey Diagrams

Sat, 01 Jun 2024 00:00:00 +0000

Sankey diagrams are powerful tools for visualizing flows of data, showing how values move from one set of categories to another. They are particularly useful for displaying energy flows, financial transfers, or user journeys etc.

A sankey diagram consists of nodes (representing categories) and links (showing the flow between these categories). The width of each link is proportional to the flow quantity it represents, making it easy to compare different flows.

Here’s as an example of sankey diagram showing the income flow (Source: App Economy Insights)

Creating Sankey Diagrams with Mermaid.js

Mermaid.js provides a straightforward way to create Sankey diagrams using a syntax similar to CSV. Here’s how you can create one:

Basic Syntax:

Start with the sankey-beta keyword, followed by your data in a CSV format with three columns: source, target, and value.

```mermaid
sankey-beta

%% source, target, value
Portfolio, Investments, 70 
Portfolio, Cash & Equivalents, 30
Investments, Equities, 50
Investments, Bonds, 20
Equities, US, 30
Equities, Others, 20
```

This will render the Sankey diagram as below:

sankey-beta %% source, target, value Portfolio, Investments, 70 Portfolio, Cash & Equivalents, 30 Investments, Equities, 50 Investments, Bonds, 20 Equities, US, 30 Equities, Others, 20

Ending

Sankey diagrams are an excellent way to visualize complex flows and relationships between different categories. With Mermaid.js, creating these diagrams becomes a simple and customizable process. You can start with a basic CSV format and enhance your diagram with various configuration options to meet your specific needs.

Info

For more detailed information and advanced configurations, visit the Mermaid.js Sankey Diagram Documentation.

The Simple Versus the Complex

Mon, 12 Feb 2024 00:00:00 +0000

In a world where the pace of change accelerates by the minute, the way we solve problems—be it in start-ups or sprawling enterprises—reveals much about our priorities, our fears, and our vision for the future.

Consider the task of generating synthetic test data, an unglamorous yet critical step in the development process.

For the start-up, the equation is simple: identify the problem, find a tool, apply it, and move forward. Faker enters the scene, a swiss army knife for data generation. It’s a quick, effective fix. Problem solved, lesson learned, onto the next challenge. This approach is a testament to agility, to the art of progress over perfection.

Then, there’s the enterprise, where complexity reigns. The same tool, Faker, becomes the seed of an elaborate ecosystem. Around it, we construct a bespoke wrapper, transforming it into a centralized service—a monument to our capability. We build dependencies, stage demos, and create roles for its upkeep. This service becomes a microcosm of the enterprise itself, a showcase for innovation, a battleground for recognition.

But as the dust settles, a paradox emerges. Despite the investment, the layers of complexity, the teams still face hurdles in accessing the very data intended to ease their journey. The quest for a comprehensive solution breeds new challenges, obscuring the simplicity of the problem at hand.

This divergence in approaches speaks volumes. The start-up, driven by necessity, opts for simplicity and speed. It thrives on the pragmatic, the straightforward solutions that propel it forward, allowing it to dance with change.

The enterprise, with its resources and its reputation, leans into complexity. It seeks not just to solve problems but to institutionalize solutions, to build monuments within its halls. Yet, in its quest for control and scalability, it sometimes loses sight of the essence of the problem, ensnared by its own creations.

What this tale reveals is a choice, not just of tools, but of paths. In the pursuit of progress, do we favor the swift and simple, or do we build cathedrals? Each approach has its place, its lessons, and its outcomes. The challenge lies in knowing which path to take, when simplicity will lead us forward, and when complexity truly serves our needs.

Exporting Book Highlights from Kobo eReaders

Sun, 28 Jan 2024 00:00:00 +0000

When it comes to eReaders, Kobo devices are my favorite. I own Kobo Libra 2 and Elipsa 2E. My Kobo library is growing with a plethora of books. I take lots of notes and highlights during my reading sessions on these devices.

I’ve always wanted to transfer all my highlights from the device to my notes. While there are numerous apps available for this purpose, I found myself searching for a more cost-effective or free solution to export the highlights. Interestingly, Kobo eReaders utilize a hidden SQLite database to store all detailing every book, highlights you’ve ever made and lots of other interesting data.

In this post, I’ll show you how to use this database and export your highlights to a markdown file. I’m keeping these files in my personal GitHub repo for future reference.

Warning

Before we proceed, a word of caution: Hacking into your Kobo device involves risks, including voiding your warranty and potentially rendering your device unusable. This guide is intended for educational purposes only. Proceed with caution and at your own risk.

Understanding Kobo Database

Kobo eReaders store a wealth of data in an SQLite database named KoboReader.sqlite. This database, hidden within the .kobo directory, contains details about the eBooks you’ve purchased, downloaded, and your interactions with them, such as annotations and highlights.

After you connect the device to your computer using USB cable, Kobo eReader will appear as one of the volumes. You can view the hidden folder and files as shown in below screenshot:

Navigate to the .kobo directory at the root of the mounted drive. Here, you’ll find the KoboReader.sqlite file.

Copy this file to your computer for safe, easy access. Do not modify or delete the file on the device!

Info

To explore the SQLite database, you’ll need an SQLite browser or manager (e.g., DB Browser for SQLite). Open the copied KoboReader.sqlite file with your chosen application to view the tables and data.

Extracting Highlights

The key to extracting highlights lies in understanding the database schema. Highlights and annotations are stored in the Bookmark table. You can execute SQL queries to retrieve this data. For instance:

SELECT ContentID, Text, StartContainerPath
    FROM Bookmark
    WHERE Text IS NOT NULL AND Text != ''
    ORDER BY StartContainerPath;

This query fetches all your highlights, organizing them by the book and the order in which they appear.

Once you’ve queried your highlights, most SQLite browsers will allow you to export this data in various formats, such as CSV or JSON, facilitating further analysis or backup.

Automating the Process with Python

Let’s create a Python program that connects to an SQLite database and retrieves a list of books with highlights, you’ll need to perform the following steps:

Connect to the SQLite database.
Query the database to get the list of books with highlights.
For each book, create a markdown file and format the highlights in markdown quote syntax.

Here’s a simple Python script to achieve the described functionality:


import sqlite3
import os

# Path to the Kobo SQLite database
db_path = './KoboReader.sqlite'

# Directory where markdown files will be saved
output_dir = './kobo_highlights'


def connect_to_database(path):
    """Connect to the SQLite database and return the connection."""
    try:
        conn = sqlite3.connect(path)
        return conn
    except sqlite3.Error as e:
        print(f"Error connecting to database: {e}")
        return None


def get_books_with_highlights(conn):
    """Get a list of books that have highlights."""
    books = []
    try:
        cursor = conn.cursor()
        query = """
            SELECT DISTINCT content.ContentId, 
                            content.Title,
                            content.Attribution AS Author, 
                            content.DateLastRead, 
                            content.TimeSpentReading
            FROM Bookmark
                INNER JOIN content ON Bookmark.VolumeID = content.ContentID
                ORDER BY content.Title;
        """
        cursor.execute(query)
        books = cursor.fetchall()
    except sqlite3.Error as e:
        print(f"Error fetching books: {e}")
    return books


def get_highlights_for_book(conn, content_id):
    """Get all highlights for a given book."""
    highlights = []
    try:
        cursor = conn.cursor()
        query = """ 
            SELECT Bookmark.Text FROM Bookmark
               INNER JOIN content ON Bookmark.VolumeID = content.ContentID
               WHERE content.ContentID = ?
        """
        cursor.execute(query, (content_id,))
        highlights = cursor.fetchall()
    except sqlite3.Error as e:
        print(f"Error fetching highlights for book {content_id}: {e}")
    return highlights


def create_markdown_for_book(book, highlights):
    """Create a markdown file for a book and insert all highlights."""
    if not os.path.exists(output_dir):
        os.makedirs(output_dir)

    title = book[1]
    author = book[2]

    filename = (f"{title}.md".replace('/', '-')
                .replace('\\', '-')
                .replace(' ', '_').lower())

    filepath = os.path.join(output_dir, filename)
    with open(filepath, 'w', encoding='utf-8') as md_file:
        md_file.write(f"# {book[1]}\n\n")
        md_file.write(f"## Author(s) -  {author}\n\n")
        for highlight in highlights:
            md_file.write(f"> {highlight[0]}\n\n")


def main():
    conn = connect_to_database(db_path)
    if conn:
        books = get_books_with_highlights(conn)
        for book in books:
            highlights = get_highlights_for_book(conn, book[0])
            create_markdown_for_book(book, highlights)
        conn.close()


if __name__ == "__main__":
    main()

This script does the following:

Connects to the Kobo SQLite database.
Retrieves all books that have highlights.
For each book, it retrieves all highlights and creates a markdown file with the book’s title as the filename.
In the markdown file, each highlight is formatted as a quote.

Here’s an output file stored in my personal GitHub Repository:

Ending

Accessing the hidden SQLite database on Kobo device opens up a new world of possibilities for managing digital reading experience. Whether you’re backing up your data, analyzing your reading habits, or simply curious about the inner workings of your eReader, the insights you gain from this endeavor are both empowering and enlightening.

Remember, while exploring the depths of your Kobo device can be incredibly rewarding, always proceed with caution to avoid unintended consequences.

Happy hacking!

AI-Assisted Programming using Open Source Models

Mon, 08 Jan 2024 00:00:00 +0000

What is AI Assisted Programming

AI-assisted programming or coding is an emerging trend in software development. This technology leverages artificial intelligence to aid software developers in various aspects of coding, from writing and reviewing code to debugging and optimizing it. AI in programming not only enhances productivity but also democratizes software development, making it more accessible to a broader range of people.

Features of AI-Assisted Programming

Code Suggestions and Generation - AI-assisted programming tools automatically suggest or generate code, speeding up the development process. It efficiently creates contextually appropriate boilerplate code and snippets allowing developers to focus on more complex tasks.
Automated Test Generation and Documentation - AI-assisted programming streamlines the creation of tests and documentation, ensuring code reliability and maintainability while saving significant developer time.
Error Detection and Correction - AI algorithms can detect errors, ranging from syntax issues to more complex logical errors, and suggest corrections. This feature significantly reduces debugging time.
Automated Code Refactoring - AI can suggest optimal ways to refactor code, ensuring it is clean, maintainable, and efficient.
Natural Language Processing (NLP) - AI programming tools understand natural language queries, allowing developers to write code using plain English, which is especially beneficial for novice programmers.
Data Analysis and Prediction - AI tools can analyze large datasets to predict outcomes, optimize performance, and automate repetitive tasks within the coding process.

Benefits of AI-Assisted Programming

Increased Productivity - Automation of repetitive tasks and intelligent code suggestions significantly speed up the development process.
Enhanced Code Quality - Consistent detection and correction of errors lead to cleaner, more efficient code.
Learning and Development - Beginners can learn coding more effectively with interactive, AI-powered tools that offer guidance and corrections.
Inclusivity and Accessibility - With AI tools that understand natural language, programming becomes more accessible to non-experts, breaking down barriers to entry in the tech world.

Commercial AI-Assisted Programming Tools

GitHub Copilot - A collaboration between GitHub and OpenAI, Copilot offers AI-powered code suggestions directly in the coding environment.
JetBrains IDEs - JetBrains offer AI-driven coding assistance with it’s suite of IDEs.
Cursor - Cursor is a popular AI-driven coding IDE built on VSCode.

Open Source AI-Assisted Programming Models & Tools

Open source large language models present several advantages as alternatives to commercial models and tools. Key benefits include:

Data Privacy - For many organizations, controlling their data is paramount. Utilizing open-source models can allow them to manage data internally without the risk of external parties accessing it.
Customization and Flexibility - These models offer flexibility for developers to train them using specific datasets. This can include applying filters on certain topics, and tailoring the model more closely to the organization’s unique needs.
Cost-Effectiveness - Open source software is generally free to use, modify, and distribute. This accessibility makes it a cost-effective solution for individuals, businesses, and organizations, particularly those looking to minimize expenses without compromising on quality and flexibility.
Collaboration and Community Development - Open source projects thrive on community contributions. They bring together diverse perspectives and skill sets, leading to more robust, innovative solutions. This collaborative spirit not only accelerates software development but also enhances its quality through continuous peer review and contributions.

mindmap root("AI Assisted Programming") ::icon(fa fa-human fa-robot) ("Models") ::icon(fa fa-cubes) ("Code Llama") ("WizCoder") ("Etc.") ("Local LLM Stack") ::icon(fa fa-computer) ("Ollama") ("Llamafile") ("Etc.") ("IDE Extensions") ::icon(fa fa-code) ("VSCode") ("CodeGPT") ("Continue") ("Jet Brains IDEs") ("CodeGPT") ("Continue")

Models

Here are some popular open source models:

Code Llama - Open-sourced by the Meta Engineering team, the Code Llama is a family of large language models for code based on Llama 2 providing state-of-the-art performance. It supports many of the most popular programming languages used today, including Python, C++, Java, PHP, Typescript (Javascript), C#, Bash and more.
Wizard Coder - Wizard Coder is another popular code generation model based on Code Llama.
Starcoder - StarCoder is a code generation model trained on 80+ programming languages.

Info

You can find the various open source models supporting code generation in Hugging Face library. Further, you can refer a benchmark of various code generation models here.

Running models locally

Here are some zero-friction tools and IDE extensions to run open source model locally.

Ollama - Ollama enables you to run a large language models on a local computer or self-hosted on Docker/Kubernetes. You can pull a model from Ollama library and get started instantly. For guidance on installing and running Ollama on your local machine, please refer to the details in my previous post Ollama - Running Large Language Models on Your Machine.
Lamafile - Lamafile is another great project from Mozilla Foundation to run models locally. You can find more about Lamafile here.

IDE Extensions

Next, it’s essential to integrate IDE extensions that link with models operating either locally or on your network. These extensions provide an experience almost like the commercial versions of coding assistants, enhancing your coding process with features like code generation and suggestions, automated test creation, documentation assistance, and support in debugging or resolving issues.

Here are some free IDE extensions that you can use with open source models.

CodeGPT - CodeGPT is a free IDE extension for AI-assisted programming available for both Visual Studio Code and JetBrains IDEs. You can find instructions to set up CodeGPT with Ollama here.
Continue - Continue is another great free IDE extension for AI-assisted programming available for both Visual Studio Code and JetBrains IDEs. You can find the model set up guide here.

Warning

The output from open source models could be under the purview of third-party licenses, including open source licenses. It’s recommended to examine the relevant license and terms of use as needed.

Building Custom Models

Building custom models based on a foundational base model, particularly using an organization’s specific codebase involves fine-tuning a pre-trained large language model (LLM) with the unique datasets derived from an organization’s own code repositories. The objective is to leverage the powerful capabilities of the base LLM - which has already been trained on vast, diverse data - and tailor it to align closely with the specific coding patterns, practices, and requirements unique to the organization.

Such an approach offers a dual advantage. Firstly, it allows organizations to harness the sophisticated AI capabilities of the base model, such as understanding complex language structures or generating code. Secondly, and more importantly, it imbues the model with a deep understanding of the organization’s specific coding environment and knowledge base. This customization leads to a more relevant and efficient AI assistant, capable of providing more relevant code suggestions, error detection, learning assistance to new joiners and other programming aids that are finely tuned to the organization’s unique technology landscape.

This results in a powerful, bespoke AI assistant that enhances coding efficiency, reduces error rates, and overall, accelerates the software development lifecycle within the organization.

Here is a high level overview of the process to build bespoke models.

flowchart TD A[Private Code Repositories] -->|Extract Code Data| B[Data Preparation] B -->|Processed Data| C[Custom Training of Base LLM] C -->|Train on Code Data| D[Trained LLM for Programming] D -->|Evaluate & Fine-Tune| E[Model Evaluation and Optimization] E -->|Finalize Model| F[Deployment of Customized LLM] F -->|Integration| G[IDE Extensions for AI-Assisted Programming] G -->|Enhanced Coding Experience| H[End User: Programmers] style A fill:#f9f,stroke:#333,stroke-width:4px style B fill:#bbf,stroke:#333,stroke-width:4px style C fill:#fbf,stroke:#333,stroke-width:4px style D fill:#bfb,stroke:#333,stroke-width:4px style E fill:#ff9,stroke:#333,stroke-width:4px style F fill:#fbb,stroke:#333,stroke-width:4px style G fill:#bbf,stroke:#333,stroke-width:4px style H fill:#9ff,stroke:#333,stroke-width:4px

Ending

AI-assisted programming is truly revolutionizing the way we write and interact with code. Its impact extends beyond mere productivity; it’s reshaping the landscape of software development, making it more efficient, inclusive, and accessible to a broader audience. As AI technologies continue to evolve, we can expect even more innovative and transformative tools to emerge in this space.

Happy AI-assisted programming!

12 Factor Docker Containers

Sat, 06 Jan 2024 00:00:00 +0000

Containerization has become increasingly popular in the tech industry, offering an efficient way to package, distribute, and manage applications. Technologies like Docker & Kubernetes, at the forefront of this revolution, provide the tools necessary to build & run containerized applications. However, creating containers can vary greatly in efficiency, security, and maintainability. Inspired by the 12-factor application methodology principles, I crafted this guide to writing Dockerfiles & building containers that stand the test of scalability and robustness. Here’s how you can apply these principles.

mindmap ("12Factor Docker Containers") ("1 Single Responsibility") ("2 Version Control") ("3 Minimal Base Images") ("4 Explicit Dependencies") ("5 Environment Abstraction") ("6 Layer Efficiency") ("7 Security Practices") ("8 Non-Privileged User") ("9 Health Checks") ("10 Build Context Minimization") ("11 Documentation and Comments") ("12 One-Time Commands")

Factor 1: Single Responsibility

Modular Containers Each Container should focus on a single concern. Avoid the temptation to bundle multiple services or applications into one container. This not only makes your container lightweight but also simplifies management and scaling.

Example:

# Good: Focuses on one application (Node.js app)
FROM node:14
COPY . /app
WORKDIR /app
RUN npm install
CMD ["node", "app.js"]

Factor 2: Version Control

Track Your Dockerfiles Treat Dockerfiles as part of your source code. Store them in your version control system alongside the application they deploy. This practice ensures synchronization between your application’s evolution and its deployment environment.

Example:

# Example of version control commands
git add Dockerfile
git commit -m "Add Dockerfile for our app"
git push origin main

Factor 3: Minimal Base Images

Opt for Slim Start with the smallest base image possible that still provides the functionality your application needs. Images like Alpine or slim variants of popular distributions keep your containers lean and fast.

Example:

# Good: Using a minimal base image
FROM python:3.9-slim

Factor 4: Explicit Dependencies

Declare with Care Clearly specify all dependencies within your Dockerfile. Avoid relying on packages present in the base image; instead, explicitly install what you need. This not only documents your application’s requirements but also prevents surprises when base images update.

Example:

# Good: Installing only necessary packages
RUN apt-get update && apt-get install -y \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

Factor 5: Environment Abstraction

Configurable Containers Externalize configuration that varies between environments (staging, production, etc.) using environment variables. This strategy avoids hard-coding settings and makes your container portable and environment-agnostic.

Example:

# Use environment variables for configuration
ENV DATABASE_URL postgres://db_user:db_pass@hostname:5432/db_name

Factor 6: Layer Efficiency

Optimize for the Cache Order Dockerfile instructions to take advantage of Docker’s layer caching. Group changes to reduce layers and rebuild times. Remember, Docker only re-builds from the first changed instruction!

Example:

# Good: Combining RUN commands
RUN apt-get update && apt-get install -y \
    curl \
    git \
    && rm -rf /var/lib/apt/lists/*

Factor 7: Security Practices

Secure by Design Employ multi-stage builds to reduce the attack surface by limiting what’s included in the final image. Regularly update and scan your images for vulnerabilities, and ensure that only necessary components are present in production containers.

Example - Regularly update the base image:

FROM ubuntu:20.04

Example - Use multi-stage builds:

FROM golang:1.15 as builder
WORKDIR /app
COPY . .
RUN go build -o myapp

FROM alpine:latest  
COPY --from=builder /app/myapp .
CMD ["./myapp"]

Factor 8: Non-Privileged User

Least Privilege Run containers as a non-root user whenever possible. This minimizes risks if your container becomes compromised and is a best practice for security in containerized environments.

RUN adduser -D myuser
USER myuser

Factor 9: Health Checks

Self-Healing Systems Implement health checks in your container. This enables the orchestrator to respond to issues, maintaining the reliability and availability of your services.

Example:

HEALTHCHECK CMD curl --fail http://localhost:8080/health || exit 1

Factor 10: Build Context Minimization

Selective Copying Use a .dockerignore file to keep the build context as small as possible. Excluding unnecessary files reduces build time and minimizes potential for unintended inclusion of sensitive files.

Example .dockerignore:

node_modules
.git
.env

Factor 11: Documentation and Comments

Clear as Day Comment your Dockerfiles thoroughly. Like good code documentation, this helps future maintainers understand why certain decisions were made. A well-documented Dockerfile is a sign of a mature development process.

Example:

# Install dependencies
# Note: libpq-dev is required for PostgreSQL adapter
RUN apt-get update && apt-get install -y libpq-dev

Factor 12: One-Time Commands

Build-time Actions Use Dockerfile instructions to run setup commands that need to only be run once, like compiling assets or code. This is more efficient than running these commands every time the container starts.

Example:

RUN chmod +x ./script.sh
RUN ./script.sh

Applying these 12 factors to your Dockerfiles & containers will not only improve the quality of your containers but will also align your development practices with modern, cloud-native principles. The result is a fleet of containers that are secure, scalable, and maintainable, ready to be deployed across diverse environments with confidence.

In conclusion, as you embark on the journey of containerization, let these 12 factors be the guiding stars that steer your Dockerfiles towards excellence.

Happy containerizing!

A simple progress bar in Python

Thu, 28 Dec 2023 00:00:00 +0000

I needed a way to display the progress of data processing program written in Python on console. I wanted to include a progress bar to show the progress. While Python packages like tqdm serve this purpose well, restrictions behind a firewall prevented me from installing and using them. As a solution, I wrote this simple progress bar from the ground up:


import sys
import time

def simple_progressbar(iterable, desc="Processing", bar_length=20):
    total_items = len(iterable)

    def print_progress_bar(completed):
        percent_complete = completed / total_items * 100
        bar = ('#' * int(bar_length * percent_complete / 100)).ljust(bar_length)
        sys.stdout.write(f"\r{desc}: [{bar}] {percent_complete:.2f}%")
        sys.stdout.flush()

    for index, item in enumerate(iterable):
        yield item
        print_progress_bar(index + 1)
    
    sys.stdout.write("\n")
    sys.stdout.flush()

# Example usage
items = [...]  # Replace with your iterable

for item in simple_progressbar(items):
    # Your actual processing logic
    time.sleep(0.1)

Story of Ada

Sat, 23 Dec 2023 00:00:00 +0000

It’s story time! Let’s hear the story of Ada and Software Reliability Engineering.

Once upon a time, in the vast and vibrant land of Technologia, there was a small village known for its skilled craftsmen and inventors. This village was unique because every inhabitant was a software engineer, dedicating their life to building and maintaining the most robust and reliable software systems in the land.

The village was led by an old and wise software engineer named Ada. She was known far and wide for her expertise in creating software that hardly ever failed. Ada had a secret, though – she was not just a skilled engineer but also a guardian of an ancient book titled “The Code of Reliability.”

This book was no ordinary tome. It contained the principles and practices of Software Reliability Engineering (SRE), passed down through generations of the greatest software minds. Ada and her fellow villagers followed these principles religiously, making their software systems near perfect.

One day, a great challenge befell the village. The kingdom of Technologia was under threat from a massive cyber-attack by a neighboring rival kingdom. The kingdom’s defenses were crumbling, and only the village of reliable software engineers could save the day.

Ada gathered her most skilled engineers, including her brightest apprentice, Linus, and set out to build a defense system that could repel the cyber-attack. They worked day and night, applying the principles from the ancient book. They focused on building resilient systems that could self-heal, scale under pressure, and adapt to unforeseen attacks.

As the enemy launched their fiercest cyber-attack, the system designed by Ada and her team stood strong. It was not just the code that was resilient but also the team’s spirit. They monitored, updated, and patched the systems in real-time, adapting to every move of the enemy.

Finally, the enemy, exhausted and unable to penetrate the defenses, retreated. The kingdom of Technologia was saved, thanks to the reliable software systems and the unwavering dedication of Ada’s team.

The king of Technologia, in gratitude, visited the village and asked Ada how they had managed to build such a reliable system. Ada simply held up the ancient book and said, “Reliability is not just in the code, but in the discipline, practices, and unity of those who wield it.”

From that day on, the village was revered across the land, and the principles of Software Reliability Engineering from the ancient book were sought after by kingdoms far and wide. Ada’s teachings spread across the world, reminding everyone that in the realm of technology, reliability is the greatest strength.

And so, the village of software engineers lived on, continuously improving and sharing their knowledge, ensuring that the digital world remains safe and reliable for generations to come.

Test Run - Using Multimodal Vision AI In Test Automation

Thu, 21 Dec 2023 00:00:00 +0000

Introduction

Multimodal models and computer vision represent a significant leap in the field of artificial intelligence, marking a transition from traditional AI systems that rely on single data types to more complex and holistic approaches. Multimodal models are designed to simultaneously process and interpret data from multiple sources – such as text, images, audio, and video. This ability to integrate diverse data types allows these models to mimic human-like understanding more closely, making them incredibly powerful in interpreting real-world scenarios and nuances.

The applications of multimodal models and computer vision are vast and varied. For example, in healthcare, these technologies can analyze medical images and patient histories to assist in diagnosis and treatment plans. In the automotive industry, they power advanced driver-assistance systems (ADAS) and autonomous vehicles, where the integration of visual, auditory, and sensory data is crucial for safe navigation. In retail and e-commerce, multimodal models enhance customer experience through personalized recommendations by understanding customer preferences through visual and textual cues.

These applications only scratch the surface of what’s possible with multimodal models and computer vision. As these technologies continue to evolve, they are set to redefine the boundaries of AI’s capabilities, offering more intuitive, efficient solutions aligned with human ways of processing information.

Using Multimodal Models In Test Automation

In this post, I’ll delve into the use of multimodal model applications with test automation, with a particular focus on UI/UX testing. This exploration into multimodal models represents a cutting-edge approach, pushing the boundaries of conventional test automation techniques. As these models continue to develop and mature, we anticipate a parallel evolution in their applications and the outcomes they can achieve. This field is dynamic and rapidly advancing, suggesting that the potential applications of multimodal models in test automation are only beginning to be realized.

Recently, I came across an exciting update from Ollama on Twitter announcing the availability of new LLaVA multimodal model, capable of processing images.

This announcement resonated with my long-standing interest in applying computer vision to testing applications. Motivated by this development, I decided to conduct a practical experiment with the model. My plan is to leverage Ollama’s platform to download and execute the LLaVA multimodal model locally, giving me firsthand experience of its capabilities in a test environment. This step represents a significant move towards integrating advanced computer vision techniques into practical testing scenarios.

Large Language and Vision Assistant - LLaVA is a multimodal model that combines a vision encoder and Vicuna for general-purpose visual and language understanding, achieving impressive chat capabilities mimicking the spirits of the multimodal GPT-4.

Use Case - UI/UX Testing Agent

Imagine we’re engaged in testing a web application using automation tools like Selenium, Cypress, or Playwright. All these tools possess the capability to capture screenshots during testing. Now, think about enhancing this process by integrating computer vision for UI/UX testing on these screenshots. The idea here is to employ a model that can provide feedback akin to a human UI/UX expert, based on specific instructions or prompts. For instance, in this scenario, I would instruct the model to analyze a screenshot as if it were a UI/UX expert, and then provide its insights or recommendations about the screen’s design and usability. This approach could significantly augment traditional automated testing by adding a visual and design analysis layer that closely mirrors human evaluation.

The following sequence diagram illustrates the concept of a UI/UX testing agent:

sequenceDiagram participant TA as Test Automation Script participant WA as Web Application participant OM as Ollama Model participant U as User TA->>WA: Test Application activate TA activate WA WA-->>TA: Render UI TA->>TA: Capture Screenshot deactivate WA TA->>OM: Send Screenshot activate OM Note over OM: Analyze Screenshot OM-->>TA: Analysis Response deactivate OM TA->>U: Report deactivate TA

Test Automation Script - The Test Automation Script using the desired tool initiates the testing process by interacting with the Web Application. This involves executing predefined test cases on the web application to assess its functionality, performance, and overall user experience.
Web Application - In response to the actions of the Test Automation Script, the Web Application renders its User Interface (UI). This involves displaying the various elements of the web application as a user would typically see them during interaction.
Capture Screenshot - The Test Automation Script then captures a screenshot of the Web Application’s UI. This step is crucial as it freezes the current state of the UI for further analysis. The capturing of the screenshot indicates a key point in the testing process where the script collects visual data.
Send Screenshot to Model - Once the screenshot is captured, the Test Automation Script sends it to the Model running in Ollama for analysis. The Model represents an advanced AI system capable of processing and analyzing visual data, presumably using computer vision and image analysis techniques.
Analyze Screenshot - Inside the Ollama Model, the screenshot undergoes analysis. This step is where the model applies its algorithms to evaluate various aspects of the UI, such as layout, color schemes, element alignments, and overall aesthetics, to provide insights similar to what a human UI/UX expert would offer.
Analysis Response - After completing the analysis, the Model sends its findings back to the Test Automation Script. This response likely includes detailed feedback or a report on the UI/UX aspects of the web application, as analyzed from the screenshot.
Report to User - Finally, the Test Automation Script relays the analysis response to the User. This step involves presenting the findings of the Model’s analysis to the user, typically in the form of a report. In this context, the user could be a test engineer, a UI/UX designer, or any other stakeholder interested in the UI/UX test results.

Running The Multimodal Model With Ollama

Ollama is an open-source tool that lets you run, create, and share large language models on your computer. I’ll use Ollama to run the LLaVA model.

Info

For guidance on installing and running Ollama on your local machine, please refer to the details in my previous post Ollama - Running Large Language Models on Your Machine

Let’s run the LLaVA model using Ollama command line:

ollama run llava

After setting up the model, let’s input a test image to evaluate the model’s response and assess its capabilities using the command line.

The model identified the image as a BMI calculator page and correctly interpreted its function.

Using the LLaVA Model In Test Automation

In this part, I will demonstrate using a Selenium script to conduct a test on a sample application, leveraging the capabilities of the LLaVA model for enhanced analysis.

The Test Automation Script

I’ve set up a Python script with Selenium to visit a BMI Calculator webpage to capture its screenshot. The script is designed to open a Chrome browser, navigate to the specified application, and then take a screenshot, which it saves as a base64 encoded string.

Info

The example code for this post is available in GitHub repo ollama-examples


from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
import base64

# Page URL
your_page_url = 'https://cookbook.seleniumacademy.com/bmicalculator.html'

# Setup Chrome options
chrome_options = Options()

chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--disable-dev-shm-usage")

# Set path to chromedriver as per your configuration
webdriver_service = Service(ChromeDriverManager().install())

# Choose Chrome Browser
# Choose Chrome Browser
driver = webdriver.Chrome(service=webdriver_service, 
                          options=chrome_options)

try:
    # Get page
    driver.get(page_url)

    # Get screenshot as base64
    base64_encoded_string = driver.get_screenshot_as_base64()

finally:
    # Close browser
    driver.quit()

Calling Ollama API In Test Automation Script

Ollama is a versatile tool designed for interacting with large language models (LLMs). It stands out for its ability to simplify complex AI interactions, making LLMs more accessible to a wider range of use cases. The Ollama API is central to this functionality, acting as a bridge between applications and the models. The Ollama API, with its well-defined set of commands and responses, enables users to effectively harness the capabilities of LLMs, facilitating a range of tasks from data analysis to natural language processing.

I’ll use the POST /api/generate API in the Selenium script. This API accepts the following parameters:

model - The name of the model required. E.g llava.
prompt - The prompt to generate a response for.
images - The list of images as base64 strings.
stream - If false, the response will be returned as a single response object, rather than a stream of objects.

To guide the model to function as a UI/UX expert and deliver feedback on the provided screenshot, we must communicate this specific role through the prompt parameter. This instructs the model on the context and perspective from which to analyze the screenshot, ensuring that the feedback aligns with UI/UX expertise.

Info

A prompt in the context of AI and machine learning, particularly with language models, refers to the initial input given to the model to elicit a response or output. Prompt engineering, then, is the art and science of crafting these inputs or questions in a way that guides the AI to provide the most useful, accurate, or creative responses. It involves strategically formulating prompts to leverage the AI’s capabilities effectively. This is crucial because the quality and nature of the output heavily depend on how the prompt is framed. Skilled prompt engineering can significantly enhance the performance of AI models, especially in complex tasks like content generation, problem-solving, or data interpretation, where the clarity and specificity of prompts can dramatically influence the results.

Let’s adjust the script to incorporate the Python requests module, enabling it to interact with the POST /api/generate endpoint of the Ollama server, which I have set up locally.


import requests
import json

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager

page_url = 'https://cookbook.seleniumacademy.com/bmicalculator.html'

# Setup Chrome options
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--disable-dev-shm-usage")

# Set path to chromedriver as per your configuration
webdriver_service = Service(ChromeDriverManager().install())

# Choose Chrome Browser
driver = webdriver.Chrome(service=webdriver_service, 
                          options=chrome_options)

prompt = """
    As a UI/UX expert, you are presented with a screenshot of a BMI Calculator page. 
    Your task is to meticulously evaluate and provide detailed feedback. 
    Focus on aspects such as the overall user interface and 
    user experience design, alignment, layout precision, color schemes, and textual content. 
    Include constructive suggestions and potential enhancements in your critique. 
    Additionally, identify and report any discernible errors, defects, 
    or areas for improvement observed in the screenshot.
"""

try:
    # Get page
    driver.get(page_url)

    # Get screenshot as base64
    base64_encoded_string = driver.get_screenshot_as_base64()
    
    # Define the API endpoint
    api_endpoint = "http://localhost:11434/api/generate"
    headers = { 'Content-Type': 'text/plain' }

    # Define the data payload with prompt and image
    data_payload = {
        "model": "llava",
        "prompt": prompt,
        "stream": False,
        "images": [base64_encoded_string]
    }

    # Send the POST request to the Ollama API endpoint 
    # running the llava model
    response = requests.post(api_endpoint, headers=headers, 
                             data=json.dumps(data_payload))
    
    # Assuming the API returns a JSON response
    response_data = response.json()
    
    # Print response
    print(response_data)

finally:
    # Close browser
    driver.quit()

This script uses the requests library to interact with the Ollama API and engage with the LLaVA model. Upon receiving the defined prompt or set of instructions, the LLaVA model will process the image and generate feedback accordingly. The output generated by the model is expected to be structured as follows, providing specific and insightful feedback based on the image analysis:

Generating Report

To conclude this example, I will enhance the script by adding functionality to create a PDF report using the reportlab module. This report will include both the screenshot of the page and the feedback provided by the model. This PDF can then be conveniently shared with stakeholders for review and further action.

Additionally, I will organize and refine the code to ensure it is modular and easily reusable, enhancing its utility for future testing scenarios.


import requests
import json
import io
import base64

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager

from reportlab.platypus import SimpleDocTemplate
from reportlab.platypus import Table, TableStyle
from reportlab.platypus import Image, Paragraph
from reportlab.lib.styles import getSampleStyleSheet
from reportlab.lib.pagesizes import letter
from reportlab.lib import colors

# A function to perform the analysis 
# using the Ollama API 
def perform_ui_ux_analysis(page_name, image):
    prompt = f"""
        As a UI/UX expert, you are presented with a screenshot of {page_name} page. 
        Your task is to meticulously evaluate and provide detailed feedback. 
        Focus on aspects such as the overall user interface and 
        user experience design, alignment, layout precision, color schemes, and textual content. 
        Include constructive suggestions and potential enhancements in your critique. 
        Additionally, identify and report any discernible errors, defects, additional features, 
        or areas for improvement observed in the screenshot.
    """
    
    # Define the API endpoint
    api_endpoint = "http://localhost:11434/api/generate"
    headers = {'Content-Type': 'text/plain'}

    # Define the data payload with prompt and image
    data_payload = {
        "model": "llava",
        "prompt": prompt,
        "stream": False,
        "images": [base64_encoded_string]
    }

    # Send the POST request to the Ollama API endpoint 
    # running the llava model
    response = requests.post(api_endpoint, headers=headers, 
                             data=json.dumps(data_payload))
    
    # Assuming the API returns a JSON response
    response_data = response.json()

    # Generate the report
    generate_report(response_data, f"{page_name}_report.pdf", base64_encoded_string)

# A function to generate the PDF report
def generate_report(json_data, filename, base64_image):
    doc = SimpleDocTemplate(filename, pagesize=letter)
    story = []

    # Decode the base64 image data
    image_data = base64.b64decode(base64_image)

    # Resize the image (width, height)
    image = Image(io.BytesIO(image_data), width=400, height=200)
    story.append(image)

    # Get a sample stylesheet
    styles = getSampleStyleSheet()

    # Convert the single dictionary JSON data to a list of Paragraphs
    table_data = []
    for key, value in json_data.items():
        key_para = Paragraph(str(key), styles['Normal'])
        value_para = Paragraph(str(value), styles['Normal'])
        table_data.append([key_para, value_para])

    # Define column widths
    col_widths = [200, 300]  # Adjust as necessary

    # Create the table
    table = Table(table_data, colWidths=col_widths)
    table.setStyle(TableStyle([
        ('BACKGROUND', (0, 0), (-1, 0), colors.grey),
        ('TEXTCOLOR', (0, 0), (-1, 0), colors.whitesmoke),
        ('ALIGN', (0, 0), (-1, -1), 'LEFT'),
        ('FONTNAME', (0, 0), (-1, 0), 'Helvetica-Bold'),
        ('BOTTOMPADDING', (0, 0), (-1, 0), 12),
        ('BACKGROUND', (0, 1), (-1, -1), colors.beige),
        ('GRID', (0, 0), (-1, -1), 1, colors.black)
    ]))
    
    # Add the table to the story
    story.append(table)

    # Build the PDF
    doc.build(story)

page_url = 'https://cookbook.seleniumacademy.com/bmicalculator.html'

# Setup Chrome options
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--disable-dev-shm-usage")

# Set path to chromedriver as per your configuration
webdriver_service = Service(ChromeDriverManager().install())

# Choose Chrome Browser
driver = webdriver.Chrome(service=webdriver_service, 
                          options=chrome_options)
try:
    # Get page
    driver.get(page_url)

    # Get screenshot as base64
    base64_encoded_string = driver.get_screenshot_as_base64()
    perform_ui_ux_analysis("BMI Calculator", base64_encoded_string)
 
finally:
    # Close browser
    driver.quit()

Upon running this script, it will generate a PDF report resembling the one illustrated below. The report will include the captured screenshot along with the analysis provided by the model, offering a comprehensive overview of the test results.

Building and deploying a custom model for UI/UX testing agent

Another impressive capability offered by Ollama is the ability to create custom models based on existing ones. This feature is akin to how ChatGPT agents can be customized. Using Ollama’s Modelfile, I’ve developed a specialized model tailored for our UI/UX Testing Agent and have made it available on the Ollama Hub.

Info

Find more about building custom models with Ollama in my blog post Ollama - Building a Custom Model

This custom model is accessible for anyone to download and integrate into their projects, providing a convenient resource for those looking to enhance their test automation processes with advanced AI-driven insights.

Modelfile


# Modelfile for the UX Doctor
# Run `ollama create theuxdoctor -f ./Modelfile` and then `ollama run theuxdoctor` and enter a topic

FROM llava
PARAMETER temperature 1

SYSTEM """
As a UI/UX expert, you are presented with a screenshot of a given page.
Your task is to meticulously evaluate and provide detailed feedback.
Focus on aspects such as the overall user interface and
user experience design, alignment, layout precision, color schemes, and textual content.
Include constructive suggestions and potential enhancements in your critique.
Additionally, identify and report any discernible errors, defects, additional features,
or areas for improvement observed in the screenshot.
"""

Ending

The utilization of AI in this context serves as a preliminary step in the testing process. It flags potential UI/UX issues, allowing developers to make early adjustments. This can significantly streamline the development cycle, as it helps identify visual and usability problems when they can be addressed more easily.

However, the nuanced understanding, experience, and judgment of a human UI/UX expert are still crucial. The final review and approval of the UI/UX design should ideally be conducted by a professional in the field. They can assess aspects that automated systems might overlook, such as the emotional response of users, the alignment with brand identity, and the overall user journey experience, which often require a human touch and understanding.

Thus, while the integration of multimodal models like LLaVA in test automation scripts enhances the testing process and provides valuable insights, it complements rather than replaces the role of human experts.

The projects, such as The Self-Operating Computer framework, are set to make significant strides in technology. This particular tool is designed to enhance multimodal models like GPT-4-vision, enabling them to execute human-like interactions with computers, including mouse clicks and keyboard strokes.

This advancement signals a move away from traditional test automation and RPA tools, paving the way for the development of artificial general intelligence agents capable of more sophisticated, autonomous actions. This shift points to a future where the capabilities of automation and AI could be greatly expanded, opening up new possibilities in various applications.

/bye

Story of Codexia

Wed, 25 Oct 2023 00:00:00 +0000

In the ancient town of Codexia, two figures were renowned for their talents: Vira, the Vigilant, a guardian of the town’s treasures, and Delvin, the Devoted, a craftsman known for creating intricate tools and machines.

Codexia’s treasure wasn’t gold or jewels, but precious lines of magical code that powered the town. But with great power came great threats. Dark sorcerers sought to exploit vulnerabilities in these codes to control Codexia.

Vira’s role was to find these vulnerabilities before the sorcerers did. She’d tirelessly inspect the code, patching weak points. However, her methods were often intrusive, requiring Delvin to halt his work, leading to frustrations.

Delvin, passionate about his creations, wished for uninterrupted crafting. He believed in the beauty of seamless artistry, where tools and machines operated without hindrance. The constant stops to accommodate Vira’s checks hampered his spirit.

One day, a wise old oracle named Orin observed their discord. He summoned them to his abode and shared a vision: “To protect Codexia, both vigilance and devotion are needed. But they must work in harmony.”

Heeding Orin’s advice, Vira and Delvin began collaborating. Delvin crafted tools that allowed Vira to inspect vulnerabilities without disrupting his creations. In return, Vira shared insights that enabled Delvin to preemptively strengthen his designs against threats.

Their combined efforts led to the creation of a magnificent shield, named “Secura,” that safeguarded Codexia’s code. With Secura, vulnerabilities were detected and patched seamlessly, enhancing both security and the crafting experience.

The tale of Vira and Delvin became legendary in Codexia. It served as a timeless reminder that vigilance and devotion, when harmonized, could create an unbreakable shield against even the darkest of threats.

And so, Codexia thrived, its magical codes safe and its tools and machines more wondrous than ever, all thanks to the united efforts of Vira, the Vigilant, and Delvin, the Devoted.

The moral of the story is Harmonious collaboration between vigilance (security) and devotion (development) is essential for comprehensive protection. Security teams creating process hindrances for development teams doesn’t help either. When diverse teams work together, understanding and complementing each other’s roles, they can create robust defenses against external threats.

Ollama - Building a Custom Model

Sun, 22 Oct 2023 00:00:00 +0000

Introduction

In the previous post of this series, I introduced Ollama, a powerful tool that simplifies the process of creating, running, and managing large language models (LLMs). It is particularly advantageous for those who want to benefit from the power of LLMs without having to deal with the complexities associated with model management. Aside from managing and running models locally, Ollama can also generate custom models using a Modelfile configuration file that defines the model’s behavior. This post explores how to create a custom model using Ollama and build a ChatGPT like interface for users to interact with the model.

The Modelfile

The Ollama Modelfile is a configuration file essential for creating custom models within the Ollama framework. It facilitates the specification of a base model and the setting of various parameters, such as temperature and num_ctx, which alter the model’s behavior. Additionally, through the SYSTEM instruction within the Modelfile, you can set context or roles for the model, aiding in more contextually aware interactions. This structured approach offered by the Modelfile allows for a clear delineation of model configurations, helping in the reproducible creation and management of custom models. By leveraging the Modelfile, you can tailor models to meet specific project needs within the Ollama environment.

You can compare the Modelfile to Dockerfile, a text document containing a series of instructions used by Docker to automate the creation and configuration of a containerized application environment. In Ollama, Modelfile automates the model creation.

Building A Custom Model - API Security Assistant

Creating a custom model in Ollama follows a structured yet flexible process that allows you to customize models according to your requirements. The process involves a series of sequential and iterative steps that build upon each other, ensuring a coherent and manageable pathway toward the creation of a custom model that adheres to the specified parameters and instructions. The below visualization encapsulates the essence of this process, showcasing the steps involved in creating, deploying, and refining a custom model in Ollama:

graph TD; A[Base Model Selection] --> B[Parameter Setting]; B --> C[System Instruction]; C --> D[Model Creation]; D --> E[Model Deployment]; E --> F[User Interaction]; F --> B[Parameter Setting]; style A fill:#f9d,stroke:#333,stroke-width:2px; style B fill:#fc9,stroke:#333,stroke-width:2px; style C fill:#9cf,stroke:#333,stroke-width:2px; style D fill:#cfc,stroke:#333,stroke-width:2px; style E fill:#ccf,stroke:#333,stroke-width:2px; style F fill:#fcc,stroke:#333,stroke-width:2px;

Base Model Selection - A base model is initially chosen, which acts as a starting point for building our custom model.
Parameter Setting - Here, certain rules are set to guide how the model should behave and respond.
System Instruction - A basic role or context is provided to the model, helping it understand how it should interact during conversations.
Model Creation - With the groundwork laid, the model is crafted using a simple command, bringing our custom model into existence.
Model Deployment - Once created, the model is made ready and accessible for interaction with a simple command.
User Interaction - Once deployed, users can interact with the model, asking questions or giving tasks based on their needs.
Feedback and Iteration - A continuous improvement cycle is established where feedback on the model’s responses can be used to make necessary tweaks, ensuring the model gets better over time.

We will create a custom model based on CodeLlama to assist developers in implementing secure APIs. This is a simple modelfile that takes and responds to any questions related to API security:

# Modelfile for creating an API security assistant
# Run `ollama create api-secexpert -f ./Modelfile` and then `ollama run api-secexpert` and enter a topic

FROM codellama
PARAMETER temperature 1

SYSTEM """
You are a senior API developer expert, acting as an assistant. 
You offer help with API security topics such as: Secure Coding practices, 
API security, API endpoint security, OWASP API Top 10. 
You answer with code examples when possible.
"""

A typical Modelfile consists of instructions and parameters delineating the model’s behavior. Here are the primary components of a Modelfile:

FROM Instruction - The FROM instruction is essential as it defines the base model that your custom model will build upon. For instance, FROM codellama in the example modelfile indicates that the custom model will be based on the codellama model.

CodeLlama
CodeLlama is a code-specialized version of Llama 2 built by the Meta Engineering team. It is tailored to generate code and natural language descriptions about code from both code and natural language prompts. By leveraging text prompts, it assists developers in creating code and understanding code snippets, making the coding process more efficient and accessible to both seasoned developers and newcomers. CodeLlama’s enhanced coding capabilities stem from additional training on code-specific datasets, part of an endeavor to create a family of large language models for code based on Llama 2. You can find more about CodeLlama here

SYSTEM Instruction - The SYSTEM instruction is utilized to set a system prompt that guides the model’s behavior during interactions. For example, setting a system prompt like SYSTEM You are a senior API developer expert, acting as an assistant can steer the model towards a particular context or role.
PARAMETER Instruction - The PARAMETER instruction allows you to set various parameters that influence the model’s behavior, such as temperature and num_ctx. Temperature controls the randomness of the model’s outputs, while num_ctx adjusts the context size.
Additional Instructions - Besides the primary instructions, you may encounter other instructions like TEMPLATE or custom instructions that further tailor the model’s behavior

To create the custom model defined in the Modelfile, execute following command:

ollama create api-secexpert

The Ollama create option will build the custom model based on instructions in the supplied Modelfile:

You can use the Ollama list option to get the list of models downloaded or created locally:

Running The Model

To run the model in the command line, use the run option followed by the model name:

ollama run api-secexpert

You can ask any questions on API security, and the model will give you an expert answer. Further models can be finetuned and trained.

Info

You can find more about Modelfile in Ollama documentation here.

Building An Interface Like ChatGPT For Custom Model

In this section, we will explore how to create a ChatGPT-like chatbot interface for our custom model utilizing two powerful tools: LangChain and Streamlit. LangChain, a library designed to interact with various Language Models and Language Model Management Systems, will serve as the conduit to our custom model, facilitating seamless interaction. On the other hand, Streamlit, an open-source app framework, will provide a user-friendly interface through which users can interact with our chatbot. The amalgamation of LangChain’s robust language model interaction capabilities and Streamlit’s intuitive, real-time app development features will enable us to create an engaging and responsive chatbot interface. Through this endeavor, we aim to demonstrate how effortlessly one can bridge the gap between sophisticated language models and user-friendly applications, thereby enhancing user engagement and experience.

What Is LangChain?

LangChain is a Python library designed to provide a streamlined interface to interact with various Language Models (LMs) and Language Model Management Systems (LLMS). It acts as an abstraction layer, allowing access and utilization of powerful language models such as OpenAI’s GPT-3 or Llama2 with a unified API, regardless of the underlying implementations or providers. LangChain aims to simplify the integration of language models into applications, reducing the learning curve and development time required to leverage the capabilities of these models.

Additionally, LangChain provides tools for managing and interacting with LLMS, facilitating language model deployment, scaling, and management. This is particularly useful for working on complex or large-scale applications where efficient language model management is crucial. By offering a user-friendly and consistent interface across different language models and management systems, LangChain helps to democratize access to advanced natural language processing capabilities, making it easier to incorporate cutting-edge language technologies into their projects. Find more about LangChain here.

What Is Streamlit?

Streamlit is an open-source app framework for Machine Learning and Data Science teams. It simplifies the process of turning data scripts into shareable web apps. With just a few lines of Python, you can create interactive reports and dashboards, real-time stream processing apps, or even complex interactive applications. Streamlit’s intuitive and straightforward API requires no front-end experience, making it a convenient choice for data scientists and engineers looking to visualize data, build interactive tools, or showcase machine learning models in a web interface. Find more about Streamlit here.

Before we build the Chatbot, we need to ensure all the necessary prerequisites are met. Ensure you have the latest version of Python installed. Install the streamlit & langchain packages.

pip install streamlit langchain

Update

You can find the code examples used in this post in my GitHub Repo.

Now, let’s create a Python file chatbot.py and copy the following contents:

import streamlit as st
from langchain.llms import Ollama
import time

# App title
st.set_page_config(page_title="👨🏻‍💻 API Security Chatbot")
with st.sidebar:
    st.title("👨🏻‍💻 API Security Chatbot")

# Initialize Ollama
ollama = Ollama(model="api-secexpert")

# Store LLM-generated responses
if "messages" not in st.session_state.keys():
    st.session_state.messages = [{"role": "assistant", "content": "How may I assist you today?"}]

# Display or clear chat messages
for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.write(message["content"])

def clear_chat_history():
    st.session_state.messages = [{"role": "assistant", "content": "How may I assist you today?"}]
st.sidebar.button('Clear Chat History', on_click=clear_chat_history)

# Function for generating Ollama response
def generate_ollama_response(prompt_input):
    string_dialogue = "You are a helpful assistant. You do not respond as 'User' or pretend to be 'User'. You only respond once as 'Assistant'."
    for dict_message in st.session_state.messages:
        if dict_message["role"] == "user":
            string_dialogue += "User: " + dict_message["content"] + "\n\n"
        else:
            string_dialogue += "Assistant: " + dict_message["content"] + "\n\n"
    response = ollama.__call__(prompt=f"{string_dialogue} {prompt_input} Assistant: ")
    return response

# User-provided prompt
if prompt := st.chat_input():
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.write(prompt)

# Generate a new response if the last message is not from the assistant
if st.session_state.messages[-1]["role"] != "assistant":
    with st.chat_message("assistant"):
        response = generate_ollama_response(prompt)
        placeholder = st.empty()
        full_response = ''
        for char in response:  # Assume response is a string
            full_response += char
            placeholder.markdown(full_response)
            time.sleep(0.05)  # Adjust the delay as needed
    message = {"role": "assistant", "content": full_response}
    st.session_state.messages.append(message)

The program creates an interactive chat interface similar to ChatGPT. Users can ask questions and receive responses from the chatbot, which is powered by the custom Ollama model. The chat history can be cleared at any time, and Ollama’s responses are displayed dynamically and engagingly, resembling a typewriter. The visualization below shows the sequence of steps that occur during the interaction:

sequenceDiagram participant User participant Streamlit participant LangChain participant Ollama User->>Streamlit: Enters question in chat_input Streamlit->>LangChain: Sends question to generate_ollama_response LangChain->>Ollama: Sends question with conversation history Ollama-->>LangChain: Returns response LangChain-->>Streamlit: Returns response Streamlit->>Streamlit: Streams response character by character Streamlit-->>User: Displays streamed response in chat_message User->>Streamlit: Enters another question in chat_input Streamlit->>LangChain: Sends new question to generate_ollama_response LangChain->>Ollama: Sends new question with updated conversation history Ollama-->>LangChain: Returns new response LangChain-->>Streamlit: Returns new response Streamlit->>Streamlit: Streams new response character by character Streamlit-->>User: Displays streamed new response in chat_message User->>Streamlit: Clicks 'Clear Chat History' button Streamlit->>Streamlit: Clears chat history in session_state.messages

The program elegantly combines Streamlit’s interactive features with LangChain’s Ollama to create a user-friendly chatbot interface, demonstrating a practical application of integrating a custom model into a web-based chat interface.

Running the Chatbot

Ensure the model is running with Ollama. Run the following command to launch the Chatbot UI:

streamlit run chatbot.py

This command launches the Chatbot UI in the browser. You can ask questions, and Chatbot will display responses from the model running in Ollama:

Ending

Ollama offers a robust and user-friendly approach to building custom models using the Modelfile. You can quickly develop and deploy AI-powered applications using custom models and build user-friendly interfaces for these models.

In the next post, we will build more advanced apps using LLM’s and Ollama.

/bye

Up and running local Kubernetes Clusters

Mon, 16 Oct 2023 00:00:00 +0000

Setting up a local Kubernetes cluster is essential for learners and developers to grasp the ecosystem’s workings and test their configurations. Tools like Minikube, Kind, K3s, and Kubeadm can be handy in creating local clusters that mimic production setups. Here is a brief review of these tools and how they can enable the creation of local, disposable Kubernetes clusters.

Minikube

Minikube is a CNCF project that facilitates the creation of a single-node Kubernetes cluster on local machines. It is ideal for learners starting their journey into the Kubernetes world. Minikube is straightforward to set up and provides a user-friendly interface to interact with the cluster.

Find more about Minikube here

Kind (Kubernetes in Docker):

Kind is a tool for running local Kubernetes clusters using Docker container nodes. It is particularly useful for continuous integration (CI) scenarios. Kind supports multi-node clusters, making it a step ahead for users who wish to simulate more realistic environments compared to Minikube.

Find more about Kind here

K3s

K3s is a lightweight, certified Kubernetes distribution designed for edge, IoT, CI, and ARM. It trims unnecessary features and configurations, making it faster and easier to set up. K3s can run on a variety of infrastructure, including local machines, making it flexible for testing and development.

Find more about K3s here

Kubeadm

Kubeadm is a toolkit provided by Kubernetes for bootstrapping clusters. While it’s not as user-friendly as Minikube or Kind, kubeadm offers a production-ready cluster setup with more configurable options. It is suitable for users with a fair understanding of Kubernetes who wish to dive deeper into cluster setup and management.

Find more about Kubeadm here

Ending

Choosing between Minikube, Kind, K3s, and Kubeadm depends on the requirements and familiarity with Kubernetes. For a straightforward setup and learning, Minikube or Kind are great choices. On the other hand, kubeadm and K3s provide more realistic and configurable environments for advanced testing and learning. By utilizing these tools, learners and developers can significantly bolster their understanding and practical skills in managing Kubernetes clusters.

Ollama - running large language models on your machine

Sat, 14 Oct 2023 00:00:00 +0000

With the increasing popularity and capabilities of language models, having the ability to run them locally provides a significant advantage to develop and research these models locally. Ollama is an open-source command line tool that lets you run, create, and share large language models on your computer.

Ollama allows you to run large language models, such as Llama 2 and Code Llama, without any registration or waiting list. Not only does it support existing models, but it also offers the flexibility to customize and create your own models. You can find the list of supported models in Ollama Library.

You can easily import the models from the Ollama library and start working with these models without installing any dependencies.

What is Large language models (LLMs)?
The Large language models are a type of artificial intelligence model based on deep learning architectures, specifically, transformer architectures like GPT (Generative Pretrained Transformer) and BERT (Bidirectional Encoder Representations from Transformers). They are trained on vast amounts of text data and can understand and generate human-like text based on the input they receive.

Getting Started

To begin your journey with Ollama, simply head over to their download page and get the appropriate version for your operating system.

Info

Note: Ollama recommends that you should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models.

Running Ollama As A Command-line (CLI)

After installing Ollama, you can run a desired model by using the following command in your terminal:

ollama run llama2

If the model is not available locally, this command will initiate the download process first. Once the model is downloaded, it will prompt for a chat with the model:

That’s it! You can start asking questions to the locally running model.

Running Ollama As A Server

Ollama can also run as a server. It has an API for running and managing models. You can start the Ollama as server using following command:

% ollama serve

This command will start the Ollama server on port 11434:

Next, you can call the REST API using any client. In this example, let’s use the curl to generate text from the llama2 model to find out who is the best batsman in the game of cricket:

curl -X POST http://localhost:11434/api/generate\
 -d '{ "model": "llama2", "prompt":"Who is the best batsman in the game of cricket?" }'

Ollama will serve a streaming response generated by the Llama2 model as follows:

We will explore this further to build a local Chatbot using Ollama REST API and LangChain.

How It Works?

If you’re familiar with Docker, Ollama works in a similar way to Docker, providing an environment where anyone can pull, test, and tinker with machine learning models similar to handling Docker images.

Pulling Models - Much like Docker’s pull command, Ollama provides a command to fetch models from a registry, streamlining the process of obtaining the desired models for local development and testing.
Listing Available Models - Ollama incorporates a command for listing all available models in the registry, providing a clear overview of their options. This is comparable to Docker’s image listing functionality.
Running Models - With a simple command, anyone can execute a model, making it effortless to test and evaluate the model’s performance in a controlled or live environment.
Customization and Adaptation - Ollama goes a step further by allowing anyone to modify and build upon the pulled models, resembling the way Docker enables the creation and customization of images. This feature encourages innovation and the tailoring of models with prompt engineering. You can also push a model to a registry.
Ease of Use - By mimicking Docker’s command-line operations, Ollama lowers the entry barrier, making it intuitive to start working with machine learning models.
Repository Management - Like Docker’s repository management, Ollama ensures that models are organized and accessible, fostering a collaborative environment for sharing and improving machine learning models.

The Ollama Runtime

Ollama offers a runtime that manages the models locally. It provides a CLI & REST API, serving as an interface for users or systems to interact with the runtime and, by extension, the large language models. The runtime enables GPU Acceleration, which would significantly speed up the computation and execution of the model. The model results, which are the output or insights derived from running the models, are consumed by end-users or other systems.

Ending

Ollama offers a more accessible and user-friendly approach to experimenting with large language models. Whether you’re a seasoned developer or just starting out, Ollama provides the tools and platform to dive deep into the world of large language models.

In the next post, we will see how to customize a model using Ollama.

/bye

Command Line Tools - jq

Thu, 12 Oct 2023 00:00:00 +0000

As a modern developer, you have access to a wide variety of tools. However, there are a few that stand out for their simplicity, versatility, and sheer power. One such tool is jq, which is a command-line JSON processor. In this post, we will take a closer look at jq, explore its basic usage, and understand how it can increase developer productivity.

What is jq?

At its core, jq is to JSON what awk is to text. It’s a lightweight, flexible tool designed to parse, query, and manipulate JSON data. Whether you’re dealing with configuration files, API responses, log files, or data payloads, jq offers a streamlined approach to handle JSON data.

jq use cases

Data Transformation - With jq, transforming JSON data structures is a breeze. You can easily reshape, filter, and modify data without the need for complex scripts or additional tools.
Quick Insights - Need to quickly extract specific data from a large JSON file? jq allows for rapid querying, helping get insights in seconds.
Automation - jq can be seamlessly integrated into scripts and automation workflows, making tasks like data validation, extraction, and transformation automated and error-free.
Debugging - When working with APIs or data-driven applications, jq can be a lifesaver. It helps in formatting and colorizing raw JSON, making it more readable and easier to debug on console.
Integration with Other Tools - jq pairs well with other command-line tools. You can easily pipe data between tools, enhancing the Unix philosophy of combining small, powerful utilities.

Installing jq

Before diving into its functionalities, let’s get jq installed:

macOS Installing jq with Homebrew

brew install jq

Ubuntu/Debian

sudo apt-get install jq

Windows Installing jq using winget

winget install jqlang.jq

Installing jq using Chocolatey

chocolatey install jq.

jq basics

Here are some basic usage examples for jq. You can try these examples in online jq playground

Prettify JSON - Simply using jq without any arguments will format and colorize raw JSON.

echo '{"name":"John","age":30}' | jq .

Working with Arrays - Access array elements with their index.

echo '["airplane", "boat", "car"]' | jq '.[1]'

This will output “boat”

Filtering Data - Extract specific values from JSON data using the dot notation.

echo '{"name": "John", "age": 30}' | jq '.name'

This will output “John”

Chaining Filters - Combine multiple filters to refine queries.

echo '{"people": [{"name": "John", "age": 30}, {"name": "Doe", "age": 25}]}' | jq '.people[] | .name'

This will output “name” from “people” array i.e “John” and “Doe”

Conditional Filtering - The select function to filter data based on conditions.

echo '[{"name": "John", "age": 30}, {"name": "Doe", "age": 25}]' | jq '.[] | select(.age > 28)'

This will select & output name “John” and age “30”

Ending

jq is a powerful tool that simplifies the often complex task of working with JSON data. Whether you’re a developer, data scientist, or just someone looking to process JSON files, jq offers a range of functionalities to make your life easier. As you become more familiar with it, you’ll discover even more advanced features and techniques to streamline your JSON processing tasks.

Find more about jq on official website

Happy jq-ing!

Designing Command Line Tools

Tue, 10 Oct 2023 00:00:00 +0000

Command line tools such as Git have become an essential part of a developer’s toolkit. When well-designed, they can greatly enhance productivity and ensure a seamless workflow. Here are some principles to consider when designing command line tools to guarantee a smooth and productive experience:

Intuitive Command Syntax - The syntax should be self-explanatory. For instance, git clone intuitively suggests that you’re cloning a repository. Avoid cryptic commands that require users to refer to the documentation constantly.
Consistent Behavior - Ensure that commands behave consistently. If a flag modifies the behavior of one command in a certain way, the same flag should have a similar effect on other commands where applicable.
Comprehensive Feedback - Always provide feedback to the user. Whether it’s a success message, a progress bar, or an error, users should always know the outcome of their command.
Error Handling with Clarity - When something goes wrong, provide clear and actionable error messages. Instead of “Error: Operation failed”, consider “Error: Failed to clone repository. Check your internet connection.”
Modularity and Composition - Design commands to do one thing and do it well. This allows users to chain commands together, similar to the Unix philosophy of piping outputs from one command as inputs to another.
Extensibility - Allow users to extend the tool’s functionality. Git’s hook system is a great example, allowing developers to trigger custom scripts at various points in the Git workflow.
Comprehensive Documentation - A tool is only as good as its documentation. Ensure that there’s a comprehensive guide available, with examples for each command and its flags.
Autocompletion and Suggestions - Implement autocompletion for commands and their arguments. This not only speeds up the process but also reduces the chances of errors. If a user mistypes a command, consider offering suggestions. For instance, if a user types git comit, the tool could suggest, “Did you mean git commit?”

For a deeper dive into designing CLI tools, Jeff D.’s article on 12 Factor CLI Apps on Medium offers a comprehensive methodology, drawing parallels with the 12-factor app principles for web applications.

In order to design a command line tool that truly enhances developer productivity, it is important to have a thorough understanding of the developer’s workflow and requirements. By prioritizing clarity, consistency, and flexibility, you can create a tool that not only achieves its intended purpose but also becomes an essential component of a developer’s toolkit.

Hello, World!

Sun, 08 Oct 2023 00:00:00 +0000

Hey there! This is my first post on my brand-new site. I’m excited to share my journey with you all!

For those curious, I’m using the following tech stack to host this site:

Hugo for site generation.
Anatole theme for Hugo.
GitHub, GitHub Pages & GitHub Actions for hosting the site and automation.
Codeapi for interactive code snippets that you can execute inside the browser, safely!

To run the code snippets in blog posts, you can hit the Run button to see the output.

print("Hello, World!")