For better or for worse, Jira is the tool that many technical organizations rely on to organize their projects. It's a great tool, no doubt, but getting visibility on projects that don't fit neatly into Jira's way of doing things can get messy. Even seasoned Jira jockeys can find it difficult to quickly get insights, and may end up building a bunch of tricks and hacks that help them do their job better. What's even more painful is manually writing reports for leadership about project status as it's reflected in Jira. Here's how we built an MVP that tackled these pain points by combining some analysis with some LLM magic.

## Recap of the problem

You may already know about [Tangential](/blog/building-an-ai-tool-for-enterprise-customers-success-and-failure) and the problem we were trying to address, but I’ll recap it here. Through our user interviews with TPMs (technical program managers) we learned that to get a good picture of a project as it exists in Jira, understanding what threatened to delay delivery, and reporting that status back to executives and other stakeholders is a very manual task. 

Apart from meetings that TPMs held to gather status and understand where the risks of a project were, they relied on Jira to get insights on what epics were at risk of not being completed, and what that would mean for the overall health of their programs.

Many TPMs we talked to had all sorts of hacks that helped ease this pain. They wrote scripts that would fetch information that dumped Jira data into a Confluence page, or would export data to a CSV that would then be loaded into a spreadsheet; one company even loaded their entire Jira database into Snowflake and built dashboards on that data. We got strong signals that they sought better visibility on their projects and to make it easier to report on issues. 

From the information we gathered through our user interviews, we decided to build an MVP that addressed these problems.

## Features for our MVP

We wanted to address the following pain points:

1. Visibility across multiple projects or programs can be difficult and time consuming to achieve
2. Writing reports that are given to executives about a program’s status is tedious, especially when different reports need to be written for different audiences

We needed to limit the scope of our MVP to only the features that would test if we could address the pain points effectively. The initial features that we would build were:

1. A way to define a project or program by entering a JQL (Jira Query Language) query and a name 
2. Ability to analyze a project by critical metrics
3. Project analysis dashboard
4. Report generation that could write a text based report for different audiences (VP of engineering, CPO, etc)
5. Ability to edit and share the text report

We believed these features would address the pain points by automating away a lot of the tedious parts of their day to day.

## Crunching numbers

To analyze a project, we had to compute some numbers that would help understand what the current status was and give clues to potential problems that could cause it to be delayed. We decided to focus on:

* Total epic story points
* Remaining epic story points
* Current velocity of the project
* Time in status (how long has a Jira ticket or epic has been in a certain state)
* Scope changes (has the number of story points increased, when, how much, and by whom)
* Changes to the epic and to its child issues (was the assignee changed, changes to description, etc)
* Comments before and after a pivot date

These calculations would be performed in code, from data fetched from the Jira API.


## How it needed to work

I started sketching out how these features would need to be built in order to work and handle at least medium-sized Jira projects. I thought this was how project analysis could work:

* User authenticates with Atlassian, providing us with a token for read access
* Web interface shown where users can define their projects via JQL query
* Project analysis is triggered by the web UI
* The backend starts by pulling in every epic from the JQL query via the Jira API and enqueuing the epic to be processed
* Using API data, each epic is analyzed in depth and a summary is written for each epic with an LLM
* The analysis is added to the database
* Once every epic is analyzed, a summary of all epics is made for the project

And on the report generation side:

* User selects a project and an audience (VP of engineering, VP of product, etc) that they want to update
* Key metrics are extracted from each epic in the project, along with the generated text summary
* LLM is prompted with this information, tailored to the selected audience based on a template
* Report is stored in the database and can be edited by the user

## Architecture

To make this work for our MVP, I decided to build the stack like this:

* **Serverless TypeScript/Node.js deployed on AWS Lambda for number crunching and analysis**

  Serverless in this use case is perfect for several reasons. I knew that the analysis of Jira data would undoubtedly take a long time and could be split into smaller chunks of work, which is where serverless can really shine. When analysis started, many lambda functions could be spun up to handle the small chunks, hitting the Jira API and doing calculations and writing to the database. 

  Serverless is also great because we didn’t need a quick response time. In this case, the analysis would be triggered by the UI and then done in the background, so it didn’t matter if there was a warmup time for the lambda functions. Also, it’s cheap to run.

  Building on AWS also naturally gives you access to all the great features needed to build a robust infrastructure, like SQS and API Gateway. When the time came to add heavy security on top of our product, that would be straightforward with AWS.

* **MongoDB** 

  I chose MongoDB because not only did they have a free hosted tier, the schemaless factor worked perfect for this use case, where there was not a great need for doing complex queries or joins across data models. The Jira analysis could be quickly stored in JSON format (basically) for fast retrieval by a key.

  They had just added vector search features when I started building this MVP, so I figured if that was something we needed in the future, we could dig in.

* **Next.js Web App/Frontend API**

  Now, I’m not really a web frontend dev. I had made some basic apps with React before, but that was a long time ago. Honestly, I was dreading making a web frontend because of how much of a hassle it was during my last experience. I figured (and hoped) that things had progressed a bit since I last had a crack at web frontends.
  
  I hadn’t used Next.js, but after having a look around at what the leading React frameworks were, this was a good fit because of its ability to include APIs and SSR (server side rendering) on top. That, and the fact that it was insanely easy to deploy on Vercel, and I was sold. I'm really happy that the tech has improved so much since I last picked it up.

* **NextAuth.js**

  NextAuth supports Atlassian authentication right out of the box, so it was easy to get going with this framework and not fuss too much with getting and storing our Jira token.
  
* **Vercel for hosting**

   I was super impressed with how slick the whole [Vercel](https://vercel.com/) experience was, how easy it was to configure different environments and integrate everything into your workflows. This allowed us to easily have a prod and dev environment setup super fast just by specifying different branches and then configuring environment variables for each.

* **OpenAI GPT-4**

  This was an obvious choice, especially for prototyping and an MVP. We did briefly consider how we could fit in other models into our product, but because it's so easy to get up and running with OpenAI, we went for it. If it became a problem later on, we would handle it then. I also already had quite a bit of experience with GPT-3/4, so that came in handy too.

* **Shared NPM package for DB access**

  Shared code makes things easier, though it can be a slight pain to setup.


* **GitHub Actions for backend CI/CD**

  CI/CD, while it's not critical to have for an MVP, really makes life easier. Generate a few GitHub actions files with ChatGPT that run tests and deploy automatically and you're good to go.


## Building it out

The first step was to build the functions that fetched and calculated the metrics that would tell us how well a project was going. To do due diligence on the Jira API and to validate that it would be feasible to crunch the numbers in the way we wanted, I first made a Python script prototype that could build an analysis for a desired JQL query. 

You may wonder why I would make a prototype of the MVP in Python if we didn't plan to ship anything written in Python. Here's the reasons why I would 100% do this again.

* **It was less up front investment.** Let's say I found out that Jira's API was missing critical features that we would need. It's hard to glean that just from the API docs, so understanding that fast by writing a cheap little script would be far better than finding that out much later.
* **It made the work a lot faster**, since I could just run the script in the command line with a hardcoded token and see the results immediately. I could then experiment with feeding the data to OpenAI to see how good the results were.
* **The code wouldn't be thrown away.** Thanks to ChatGPT, I could just throw the prototype script in and ask it to convert it to Typescript, with almost no problems.

If you're interested in checking out the Jira analysis functions, you can find them [here](https://github.com/akfreas/tangential-backend/tree/main/tangential/src/utils). 

## Ok, we can do this!

After doing a proper technical validation with the script, I moved ahead with building the foundation of the app. We ended up with a simple but solid architecture for the MVP.

![](https://a-us.storyblok.com/f/1018485/2018x1981/34f8b38367/tangential-architecture-tangential-arch-diagram.jpg)

[The backend code is available on GitHub](https://github.com/akfreas/tangential-backend)

Stay tuned for:

* How to break down analysis into smaller tasks
* How I made the analyzer handle huge Jira projects
* How to prompt GPT-4 with enough information to get what you want without blowing up the prompt

Thanks for reading and see you soon!

Getting Insights from Jira With LLMs Part 1: Architecture

Analyzing Jira Data With LLMs - Building the MVP

I love using OpenAI’s **Custom GPTs** to solve little problems I have and make life a bit easier. In a matter of minutes, you can do the same, usually with no coding required.

## What are Custom GPTs?

Custom GPTs are essentially customized ChatGPT interfaces that work exactly like ChatGPT but have special context added that helps it achieve the purpose of the GPT.

This feature was rolled out by OpenAI back in November of 2023, and it’s a feature in ChatGPT which is essentially a way to do prompt injection for each chat message you send to the GPT. That’s a fancy way of saying that it will start your chat session as if you had pasted in some text before you write your actual prompt to ChatGPT. 

It can do more than just that. You can have your custom GPT call APIs, reference knowledge that you’ve uploaded, and perform calculations. It will even suggest a name for the GPT and create a logo for it.

You can create your own custom GPTs by going [here](https://chat.openai.com/gpts/editor).

## What can I do with Custom GPTs?

I honestly don’t think that OpenAI does a good enough job of explaining the use cases for this feature, so I’ll give some concrete examples of problems I’ve solved using this.

### Creating classified listings from an image

**Problem:** I’ve got a lot of stuff I want to sell on Kleinanzeigen (the German Craigslist), but writing out descriptions of the items can be really tedious. Especially if I want to write them in German.

**Solution:** Listmeister, a custom GPT that you can upload a photo of your item to. It will identify the item and ask you any questions that it thinks that it needs answered in order to write the best listing possible. Then, it outputs the description of the item in German, along with a catchy title.

**Result:** I can easily list items on Kleinanzeigen at insane speed. Getting a dozen or so listings up takes less than 20 minutes now.

[Link to ListMeister](https://chat.openai.com/g/g-SDSvT5EXl-listmeister)

### Digitizing recipes from an image

**Problem:** I use an app called Paprika 3 to manage recipes. Though this app does a great job of taking recipes from online sources and processing them for use in the app, I find that the best recipes are contained in the cookbooks I have. Entering the recipe details into the app manually is really tedious.

**Solution:** PaprikaGPT. This custom GPT takes an image of a recipe and extracts all the recipe data, compiles it into a JSON format that can be read by the app, and then calls an API to zip and prepare the file for consumption by the app. 

**Result:** I can quickly and accurately get a recipe from a cookbook into the app so I don’t get the cookbook dirty and can instantly add the items to the app’s shopping list automatically. 

[Link to PaprikaGPT](https://chat.openai.com/g/g-2LoGMOyB7-paprikagpt)

I was able to make Listmeister without any coding, and from when I had the idea to when I was able to use it took about 15 minutes. I just followed the wizard in the “create” section of the editor and told it what I wanted it to do. 

PaprikaGPT was a bit more involved, since I had to make an API to process the JSON into a proprietary format. But it was still pretty quick to make, and definitely will save me a lot of time as I’m digitizing recipes.

These are just some examples of how you can take your ChatGPT experience to the next level.

What are Custom GPTs?

What are Custom GPTs and How Can I Use Them?

I just translated my iOS app into 5 different languages for less than $2.

Are you looking to localize your iOS app, but don't want to spend a bunch of money doing it? Cool. You're in luck, because a script I wrote, powered by OpenAI, can make this process super easy and cheap for you.

## Why Localization is Crucial for Your iOS App

Localizing your app is a key step in reaching a wider audience. By translating your app into multiple languages, you're not just expanding your user base; you're also able to enter app stores with potentially less competition for your ideal user. This strategy can really increase your app's visibility and user engagement, especially for iPhone users searching for apps in their language. The goal of any app developer should be to ensure that their product is accessible and user-friendly to a global audience, and localization can play a big role in achieving this.

Unfortunately, localization options can be quite expensive, running up to hundreds of dollars per language, and can cost a lot more if you've got a lot of strings. This might not be a great option for independent developers or small businesses. Fortunately, there's an AI alternative!

## Harnessing the Power of Machine Translation

With advancements in AI, automatic localization of an iOS app can be done with just a few commands on the command line. Machine translation has improved significantly, especially when the entire context is provided. This makes it a viable option for app localization.

## Your Guide to Affordable iOS App Localization

Here's how you can localize your iOS app swiftly using my script and an OpenAI API key:

**Use Xcode's .xcstrings Format**: Make sure you're using Xcode's new .xcstrings format for your localizations, as announced at WWDC 2023. This modern strings format integrates a built-in UI for easy modifications.
   
   Haven't switched yet? Migrating from Localizable.strings files is straightforward. Learn how here: [WWDC 2023 Session](https://developer.apple.com/videos/play/wwdc2023/10155/).

**Clone the GPT-Localize-iOS Project**: Visit [GPT-Localize-iOS](https://github.com/akfreas/GPT-Localize-iOS) and clone the project. You'll need an OpenAI API key, which you can obtain [here](https://platform.openai.com/docs/quickstart).

**Set Up and Use the GPT-Localize-iOS Script**: 

In order for this to work best, you should add comments to each localization key for better translation outcomes. This will help GPT-4 to understand what it's translating and where the string appears.

First, you need to get an OpenAI API key [here](https://openai.com/api/). Then, set the API key as an environment variable `OPENAI_API_KEY` or enter it when prompted by the script.


I recommend using a virtual environment (venv or virtualenv) for dependency isolation. Install the dependencies with `pip install -r requirements.txt`.

Run the script with Python 3.x. Here's how you would translate an `xcstrings` file from English to Spanish:
  
    python3 translate_strings.py \
    --input-file=path/to/your/file.xcstrings \
    --target-language-code=es \
     --source-language-code=en
      
[](https://a-us.storyblok.com/f/1018485/1218x442/860bdb8bc2/screenshot-2024-01-30-at-17-54-44.png)

You'll see how much it will cost you and the progress right in the command line. Check out the Readme for more flags and options for using this script.

**Disclaimer**: The tool uses OpenAI's API and may incur costs. Check [OpenAI pricing](https://openai.com/pricing/) for details.

## Go Global with Confidence

With this approach, you're not just translating; you're localizing your iOS app with precision and cultural relevance, essential for engaging a global audience. Taking advantage of Xcode's new localization features and combining them with advanced AI will give you the opportunity to expand your app's reach without breaking the bank.

Viel Spaß and happy coding as you take your iOS app to new heights and markets!!


A Guide to Localizing an iOS App For Less Than $2

Last June, Akio Nuernberger and I met on YC Cofounder Match as I was looking for someone to explore new ideas and build a product with. He and I hit it off and quickly formulated an idea for a product: a tool that allowed technical leaders to get better visibility on their projects. We started exploring the idea space by arranging user interviews with leaders who we thought would benefit from this product. It took a few turns: we talked to product and engineering managers first, and after talking with an engineering manager from Snowflake, we were tipped off to check out how Technical Program Managers (TPMs) were getting visibility on their projects, since they work more across teams and manage multiple projects at once.

## Talking & Learning
I could write a lot about how the journey progressed from here, but I’ll share with you the key moments. First, we learned about the role of the TPM. It wasn’t a role that I was personally familiar with, though Akio knew more than I did. If you're not familiar with what a TPM does, they oversee the delivery of complex technology projects; coordinating resource allocation, and communicating with technical and non-technical stakeholders to ensure that everyone is on the same page and that risks are being addressed and mitigated. 

Then, after interviewing a few TPMs, we discovered that they did a lot of manual work to gather information from different sources to compile reports that were given back up to stakeholders (VPs, C-suite, and other managers). Many times, the report would need to be adjusted for different audiences, and many of the TPMs that we talked with already had technical hacks to solve this problem. Awesome! We were onto something, a real problem that we could solve with a tool! Our hypothesis was that a tool that automatically gathered data from Jira (to start) and ran it through an LLM would save a huge amount of time each week for TPMs. We also believed that focusing in the beginning on TPMs as a niche would be a good starting point, and that the product could easily serve other technical leaders in the future, as well.

We created a pipeline of LinkedIn TPM profiles to auto connect with to get user interviews and ended up talking to over 70 people in the field.

## The Interviews
Following Rob Fitzpatrick's "Mom Test" strategy for interviewing users in this phase, we only asked the interviewee about their day-to-day and the pain points that surfaced. I think we did a pretty good job with the interviews, despite having a strong idea of what we wanted to build. However, I do think this narrowed our exploration of the problem space slightly, since we were motivated to validate the idea we already had, rather than press on other pain points that we might have been able to address. Of course, it's hard to do all this in a 30 minute user interview, but I would try to keep more of an open mind the next time I found myself in this phase.

After we had done quite a few interviews, we built a mockup in Figma and started showing it to the people we had initially spoken with. Some were very excited about what we were going to build, which was encouraging and helped us move forward.

Now, most of the people we had interviewed were from larger companies (500+ headcount) that required a TPM in the first place. Because our idea would involve getting read access to Jira, we would need to clear a pretty large hurdle: security in the organization. However, we believed at this point that if the problem we were solving with our tool was big enough, the motivation would be there for the TPMs to help us clear this. So, we started to build.

## Finally, Time to Build (Right?)
I worked for about four weeks building a functional MVP that connected with Jira, could do analysis on projects via a JQL query, and write text status reports using that data. The backend is a Serverless Node.js app written in Typescript, and the frontend is a Next.js React app deployed on Vercel. It pulls in epics from JQL queries and analyzes them for important metrics, including:


- Velocity
- Recent changes to Epic scope
- Time in status
- Predicted end time (overdue or on-time)
- Changes that may present a risk (assignee, story point changes, etc)

Then, the metrics would be fed into GPT-4 to generate a textual summary of what was happening in the epic, which would show up in the web dashboard:

![Dashboard with Tangential Jira](https://a-us.storyblok.com/f/1018485/2232x1135/223c52b144/tangential-dashboard.webp)

This enabled quick insights into how the epic was progressing, and by computing metrics that were relevant to reporting status, we could feed these into the report generation feature. 

Getting insights on epic progress wasn't the primary feature that we were aiming for. Jira does this pretty well already, even if they are missing some key features that would be useful for anyone trying to get a higher-level view of progress. The analysis was used to address the main pain point that we had in mind: generating status reports for executives and other stakeholders.

![Report Generation](https://a-us.storyblok.com/f/1018485/1181x1000/24dffbf010/1704889547559.png)

The report generation took all the metrics computed in each project, as well as their summaries, and used GPT-4 to write a status report, showing the epics that were at risk as well as possible actions to be taken. The idea was that it would serve as a starting point for something to be posted in Slack or sent out via email.

![A Tangential generated report](https://a-us.storyblok.com/f/1018485/923x1000/3b4e14b01f/tangential-report.webp)

Even when using our own Jira with a small number of data points, we thought it worked quite well. It could easily handle large epics and projects by breaking them into smaller chunks of work to be done asynchronously. I personally had a ton of fun building this and it came at the perfect time, since GPT-4 Turbo with the 128k context window had just come out, making it easy to generate full-fledged reports based on a lot of data. It was a proper MVP and something that we could go back to the TPMs we talked to initially, which is what we did.

Hey, We've Got Something!
This is where we got our first big, concrete blow. We approached the TPMs who said they were interested, and many responded with “I’ll talk with my manager about it” or “I think it’s going to be really difficult with our security team”. We reflected more, talked with a few more people, then realized that the problem we were solving wasn’t urgent or needed badly enough. As we were realizing this, we started reaching out to TPMs in smaller companies with our MVP proposition. We were met with almost complete silence. This is when we decided that, while our tool might be useful to some, it didn’t solve a problem big enough to be a viable business.

## Reflecting Deeper

Upon reflecting further on what we were doing, we realized a few more key things. First, the task that we thought we could automate, while it can be a manual and laborious one, is actually something that helps TPMs build trust and communicate with stakeholders. Gathering the information through Jira can help a bit, but much more can be gathered through 1:1s and other direct communication. Knowing when something is going to fall behind schedule is a critical part of the job, and often Jira won’t reflect accurately what is really going on with a project or team. It can be a starting point, but having that culture of trust and honest communication will get better results than data on a spreadsheet.


We learned a lot during this journey:

- We should have started with zeroing in on a problem, not with selling a solution.
- To think twice about using automated outreach for finding potential users. Many of the TPMs that we got responses from were larger companies, which we might have been able to discard if we thought deeper about the security requirement.
- Validate by getting commitment to use or buy before building anything. We might have gotten better signals about commitment if we approached smaller startups right off the bat.
- Use negative hypotheses when ideating to explore what might already be a dealbreaker (in our case: security issues & the fact that automation might not be ideal because it has second-order effects)
- Know when to stop: doing so many user interviews felt like we were making progress but it didn’t help us make as much progress as we thought we were making. It was a kind of "fake" validation that made us feel good about our direction.

All this will come in handy moving forward, and I’m happy we got the opportunity to gain this experience.

I’m also very grateful to all the people we met along the way. We met some fantastic leaders at world-class companies, and it was really satisfying to learn how they are able to deliver incredibly complex projects effectively. If you are reading this and were one of these people, Akio and I really appreciate the time you took with us.

## We've Open Sourced the MVP
We decided to share the code for the MVP we built with everyone! You can find it on GitHub:

[Tangential Backend (Node.js/Typescript)](https://github.com/akfreas/tangential-backend)

[Tangential Frontend (React/Next.js)](https://github.com/akfreas/tangential-frontend)

[Tangential Core - Shared Functionality (Typescript)](https://github.com/akfreas/tangential-core)


Cheers, and here’s looking forward to the next venture!

Blog

Recent Posts about Startups and All the Tech I Love.

Getting Insights from Jira With LLMs Part 1: Architecture

What are Custom GPTs?

A Guide to Localizing an iOS App For Less Than $2

Building an AI Tool for Enterprise - Reflections on this Journey