Gift Genie: an agent that doesn't save you time

This is a guest post from winners at our recent hackathon.

We built GiftGenie with a thesis in mind - that agents can provide value beyond saving people time. We believe that not everyone cares to optimize every aspect of their life. Some people enjoy savoring their unique moments, defining for themselves a life well-lived, a life that is meaningful and unique in its own way. Some don’t mind spending more time taking the longer path, and smelling the flowers along the way.

One of the ways we as humans define a meaningful life is to cultivate our relationships. Gift-giving is one of the five love languages, a key way in which we express affection. It’s an ancient, deeply human practice. But planning a business around it, especially one that doesn't optimize for time, defies conventional wisdom. How do you calculate customer lifetime value when the goal isn't efficiency? What’s the market size for people willing to spend more time selecting gifts? The degree of need for humans to express affection is immeasurable, but we saw an opportunity at the LlamaIndex Ragathon to test our hypothesis.

A big bold bet we made with this project was the possibility that indeed there is a fundamental human need for expressing affection, and in fact, that there is a huge market of people who are invested in putting in additional time into the gift-giving process. There is no numeric proof through traditional metrics collection, but LlamaIndex Workflows can aid with getting a working prototype up and running quickly for user testing.GiftGenie is not about saving people time, it’s about making gift-giving more thoughtful.

The Heart of GiftGenie

We recognized that by making gift-giving a mindless robotic task, we would be taking the heart out of gift-giving - the time taken to think about someone. Do you remember receiving a good gift and feeling appreciated as a result, knowing that there was thought that went into it? That’s why we wanted to encourage and support people in their gift-selection journey, aiding the normally single-threaded process between the user versus themselves by providing another vantage point.

We also believe that by imagining how we look through trying on that shirt, touching that fuzzy sock, smelling that candle - precisely through these world-exploratory experiences in the act of shopping, that we help understand ourselves better. Shopping can be a sensory indulgence, a visual feast and a multisensory discovery. We recognize and celebrate it. Spending time to identify what we like and don’t like and thinking through what we could gift our loved-ones are both formative experiences towards self-discovery as well as a way of cultivating ourselves to be more attuned to the people around us.

A Separation-of-Concerns Approach

Instead of following the conventional path of browsing e-commerce sites, we wanted to experiment with using well-defined function calls explicit to the series of clearly defined steps necessary to solve this task. In this way, the developer controls the recommendation system and not the e-commerce platform involved. E-commerce platforms are limited in their bias towards their product display and recommendation strategies whether it is by number of items sold, profit margin and/or human attention span. When shopping by multiple item categories, an e-commerce webpage will not be able to reliably present top recommended items per interest category consistently organized by item categories.

In our minds when selecting a gift, we are thinking about whether or not someone would like the item, imagining how our recipient may utilize the gift in positive beneficial ways, or on the contrary being “stuck” with another capitalist byproduct that they are responsible for safekeeping. In this “mulling over” or “internal tug-of-war”, one would benefit more from watching a debate on the positives and negatives on how the product would be received because that’s the way we think, and that is how we built this app.

How does it work ?

We mapped the user’s interest based on tweets extracted using the Toolhouse API and text input into item categories, then item categories to multiple gift selections per category. These gift selections can then be debated over for appropriateness. It would be an option for the developer to run debates in parallel, as well as have the debate evolve into another gift idea. This kind of fine-grained control over the picking of slight variations of items over a handful of separate gift categories would be similar to having multiple e-commerce website tabs open searching one specific item at a time in parallel.

Our final selections after reasoning over the debate involved selecting the top item from an Amazon keyword search using the Apify API, but could in the future evolve into receiving top n results, including recommendations presented on the website, and then performing further iterations of debate and gift selection in a loop.

As you can see, when there is a tool that can condense specific actionable steps into modular building blocks, the sky can be the limit in terms of your creativity.

The GiftGenie's Nine-Step Process

Here’s a breakdown of the technical pipeline that powers GiftGenie:

1. Tweet Extractor: Collects relevant data (tweets) from a user’s social media handle or accepts text input if the user doesn’t have an account.
2. Tweet Analyzer: Interprets the data to identify interests and preferences.
3. Interest Mapper: Translates the user’s interests into potential gift categories.
4. Gift Idea Generator: Proposes specific gift ideas based on those categories.
5. Mediation Process: Initiates the first argument for each pro and con agent in the debate process setting the rules and tone of the debate.
6. Gift Debater: Argues for and against each gift idea 3 times to refine the selection.
7. Gift Reasoner: Judges the debate’s outcome based on uniqueness, practicality, and alignment with the recipient’s preferences.
8. Amazon Keyword Generator: Creates search terms to help users find the selected gifts.
9. Amazon Product Link Generator: Retrieves product links and displays them to the user, complete with images and prices.

Each stage in the gift-selection process is represented by a specific event class and corresponding function decorated with the @step decorator.

We used the Toolhouse API to extract the latest up to 10 tweets from the X account. The tweets are then categorized into interest topics fitting for e-commerce.

The event-driven architecture allows us to modularize the process and easily add or modify steps. For example, in the Tweet Analyzer, we ensure at least five interest categories are identified—Technology, Self-Care, Travel, Food, and Fitness as fallback categories—if user data is limited.

We then map the interests into product categories in a dictionary format. Some examples we had in our prompt are: Self-Care:Organic facial serum, Entertainment: Independent cinema movie tickets and Technology: High-quality charger cable.

In the next step the mapped interests to categories are reviewed to further generate items that may be good for gifts. Preferences were given to local artisans, small businesses, or specialty shops. Unique items not commonly found in regular stores are also preferred. Some fallback items at this stage are: specialty dark chocolate, premium coffee, charcuterie board items,perishable boutique pantry items, charcuterie, fruit preserves, fancy olive oil, specialty nut butters.

Sample product category to gift item for someone who tweets about TV shows:

Smart Home Devices: ['Wyze Cam 1080p HD Indoor Wireless Smart Home Camera', 'Smart plug, Gosund Mini Wifi Outlet']

Concert Tickets: ['Gift Card for a Concert', 'Live Nation Gift Card']

Music Posters: ['Pink Floyd Poster', 'The Rolling Stones Poster', 'The Beatles Poster']

After the possible gift items have been defined, we instruct an LLM to generate in separate requests the first pro and con arguments for and against the gift. The “mediator” lets the negative argument go first against the gift, focusing on limitations and drawbacks of the item, and to have the LLM provide specific reasons why the recipient might not appreciate the gift. Some examples of the outcome may be a lack of use of the item, or if an item may seem offensive, such as exercise equipment given to send a message that the recipient is out-of-shape.

The Gift Debate: Adding Depth to Decision-Making

Sample con-agent argument:

'con': "A gourmet cheese basket may not be a suitable gift for all. The recipient might be lactose intolerant, vegan, or simply not a fan of cheese. Additionally, gourmet cheese often requires proper storage conditions which the recipient might not have." 

One of the most intriguing parts of GiftGenie is our gift debate system. The process mirrors human decision-making, weighing potential benefits against drawbacks. Here’s how it works:

• The debate starts with a negative argument, highlighting why a gift might not be appreciated.
• A series of back-and-forth rounds between pros and cons follow.
• After three rounds, the debate’s key points are distilled into a summary.
• A final reasoning step judges the gift based on criteria like uniqueness, practicality, and alignment with the recipient's interests.

By incorporating these debates, we simulate the thoughtful consideration people naturally go through when selecting a gift. It ensures that GiftGenie doesn’t just offer quick solutions but rather thoughtful, well-rounded suggestions.

Once the debates conclude, we move to product discovery using the Apify API. Gift ideas are converted into Amazon search terms, and we retrieve product links, presenting them to users with images and prices.

So how does LlamaIndex Workflows work with our project?

With LlamaIndex Workflows, each distinct action that we wanted to take had an 1:1 relationship with event class names like this:

class InitializeEvent(Event):
    pass

class TweetAnalyzerEvent(Event):
    tweets: List[str]

class InterestMapperEvent(Event):
    interests: str

class GiftIdeaGeneratorEvent(Event):
    gift_categories: str

The workflow is orchestrated through these event transitions:

@step
async def initialize(self, ctx: Context, ev: StartEvent) -> TweetAnalyzerEvent:
    # Processing...
    return TweetAnalyzerEvent(tweets=processed_tweets)

@step
async def tweet_analyzer(self, ctx: Context, ev: TweetAnalyzerEvent) -> InterestMapperEvent:
    # Processing...
    return InterestMapperEvent(interests=str(interests))

Each @step decorator defines a method that takes the previous event and returns the next event. The type hints (ev: TweetAnalyzerEvent -> InterestMapperEvent) define the flow. The workflow automatically knows which step to call next based on the event types

Context persists throughout the workflow and is used for storing LLM instances, agents, and intermediate results.

Working with LlamaIndex Workflows: Key Takeaways

Using LlamaIndex Workflows aids in orchestrating these complex multi-step processes by distilling event orchestration into just names and types. Are you passing in any context? Yes or no? Python’s type hinting is already well-established, so it would be intuitive to leverage it for passing event input and outputs. LLM’s understand class inheritance and how decorators work, so they are also able to comprehend it and generate code effectively.

class MediationEvent(Event):
    gift_ideas: List[str]

class GiftDebaterEvent(Event):
    gift_ideas: List[str]
    debates: Dict[str, Dict[str, str]]

@step
    async def mediation_agent(self, ctx: Context, ev: MediationEvent) -> GiftDebaterEvent:
return GiftDebaterEvent(gift_ideas=ev.gift_ideas, debates=debates)

Future Directions

After completing GiftGenie, we’ve identified several areas for future exploration:

• Run Steps in Parallel: With LlamaIndex’s Workflow Async design, multiple events can run concurrently. We can experiment further with different ways to orchestrate steps in parallel.
• More Data Sources: We can incorporate other data sources such as Instagram, Youtube or Amazon Registries to further personalize recommendations.
• Incorporating Human-in-the-Loop Interventions: Based on an initial set of recommendations, users can further input their preferences and run another product-discovery-debate-reasoning loop to refine their product picks.

Let's connect!

Agents, by their nature of bundling multiple function-calling steps, might not be as fast or efficient as traditional means. We let the agent own their unhurried pace by employing it as a prototyping tool to mirror the complex, human decision-making process of gift-giving, prioritizing meaning over speed. If you build it, will they come? - is the question we can leverage Workflows as a tool to find out. In doing so, we hope to offer other builders some food for thought to help spark ideas on what they might want to build with LlamaIndex Workflows.

Feel free to check out our project on Github. We were also on a podcast with Toolhouse AI. If this project resonated with you, let’s connect and discuss it! You can find us on LinkedIn: Tanmesh and Lily.