Real-world applications often require access to dynamic and constantly changing information. This tutorial focuses on integrating external data sources, specifically APIs, into Vertex AI Agent Builder, unlocking possibilities for truly intelligent conversational experiences. By doing so, your agents can access real-time information, stay relevant, and provide accurate responses.
2. Understanding the Basics
What is Vertex AI Agent Builder?
Vertex AI Agent Builder is a platform for creating AI agents capable of interacting with and understanding the world. It supports natural language queries, integrates enterprise data, and becomes even more powerful with external data sources like APIs.
Static vs. Dynamic Data
Static data, such as PDFs or CSV files, quickly becomes outdated, limiting its use. For example, an e-commerce support agent relying on static data can’t provide live updates on product availability or shipping timelines. Dynamic sources, like APIs, offer real-time information, enabling agents to respond effectively to evolving needs.
3. Preparing for Integration
API Setup
To illustrate dynamic data integration, we use a Warehouse Address API. This API retrieves a warehouse’s address based on its ID, offering real-time updates. It is implemented using Flask and deployed on Cloud Run.
The configuration is defined in a YAML file, which acts as a blueprint for the API. It includes details like API paths, request parameters, and response formats. You can check the YAML file given below:
openapi: 3.0.0
info:
title: Warehouse Address API
version: v1
servers:
- url: 'https://my-warehouse-api-zycua53bta-uc.a.run.app'
paths:
/warehouses/{warehouseId}/address:
get:
summary: Get Warehouse Address
description: Retrieves the address of a warehouse by its ID.
parameters:
- in: path
name: warehouseId
schema:
type: integer
format: int64
required: true
description: The ID of the warehouse.
responses:
'200':f
description: Successful response with the warehouse address.
content:
application/json:
schema:
type: string
description: The full address of the warehouse.
'404':
description: Warehouse not found.
This YAML file is crucial for integrating the API with Vertex AI Agent Builder.
Connecting Your API
Open the Agent Builder console.
Navigate to Tools and create a new tool.
Enter the tool’s name, type (OPENAPI), and description.
Paste the YAML file content and save it.
Proceed to create your agent.
4. Building and Testing Your Agent
Agent Setup
Create a new agent in Vertex AI.
Define its name, goal, and step-by-step instructions.
Reference the tool created earlier using the format: ${TOOL: Tool_name}.
Test your Agent
Just like any software, agents need thorough testing. This helps identify bugs in their code, understand their current capabilities, and most important, simulate real interactions with users to refine the overall customer experience.
While you are in the console, go ahead and select the appropriate agent and select the generative model of your choice and you now ask questions to the agent and the response will be coming from the API.
5. Publishing the Chat App
Now that the Chat App is up and running, it’s time to publish it and embed it into our website. To publish your Chat App, click on the Publish button which will prompt you with the following:
You have to specify the agent environment, followed by the Access type and the UI Style that you want. Once you have all of this set, go ahead and click on Enable the unauthenticated API.
6. Conclusion
This tutorial demonstrated how to integrate external data sources into Vertex AI Agent Builder, using APIs to create dynamic conversational agents. By following these steps, you can build intelligent chat applications that adapt to real-time data and deliver superior user experiences.
We are thrilled to announce that Ryan Brady, Premier Cloud’s VP of Sales, has been recognized as one of CRN’s Next-Gen Solution Provider Leaders for 2024!
This prestigious honor spotlights young professionals under 40 already making a significant impact within the IT channel. Ryan’s drive, vision, and dedication to innovation have played an instrumental role in shaping Premier Cloud’s strategic direction and elevating our position in the marketplace.
Key Accomplishments That Set Us Apart
Over the past year, Ryan has led numerous initiatives that have dramatically diversified Premier Cloud’s product and service portfolio, creating multiple new revenue streams. We are proud to be a top seller in North America for sales of Google’s new Gemini product. This is a testament to our ability to stay ahead of industry trends and meet market demands for cutting-edge AI solutions.
Under Ryan’s leadership, we’ve also attained Premier status within the Google Cloud Partner Program and received several Specialization awards for our expertise in delivering Google Workspace and Google Cloud services. These accomplishments not only validate the exceptional quality of our solutions but also highlight our team’s dedication to excellence in every client engagement.
The Role of AI in Our Growth Journey
Artificial intelligence is transforming every industry, and Premier Cloud is at the forefront of this shift. As a reseller of AI solutions, particularly Google’s Gemini for Google Workspace and Vertex AI, we are uniquely positioned to help companies explore, understand, and implement AI meaningfully. Ryan’s vision for AI has enabled us to become trusted advisors in this space, helping clients bring their AI aspirations to life.
Join Us in Celebrating This Milestone
Ryan Brady’s inclusion in CRN’s Next-Gen Solution Provider Leaders list is a significant achievement, not just for him but for everyone at Premier Cloud. It’s a testament to the hard work, dedication, and innovative thinking that defines our team and drives our success.
We look forward to building on this momentum as we transform the IT channel, empower our clients, and push the boundaries of what’s possible.
As AI continues to evolve, Google’s Gemini AI models, developed by DeepMind, have made it easier for everyone—from individuals to businesses—to benefit from powerful AI tools. Whether you’re looking to speed up simple tasks or take on complex projects, there’s a Gemini model suited to your needs. In this guide, we’ll walk you through the key Gemini models and help you decide which one is right for you.
Meet the Gemini Family
The Gemini models are designed to handle various tasks, from quick, real-time responses to more complex, data-heavy applications. Below, we’ve outlined three of the main models—Gemini 1.5 Flash, Flash-8B, and Pro—so you can quickly see which one fits your requirements.
Quick Comparison of Gemini Models
We’ve made it easy to compare the key features of each model in the table below. Whether you’re after speed, complexity, or enterprise-level performance, this table helps you pick the right option.
What Are Tokens and Why Do They Matter?
Tokens represent pieces of the input or output text that the model processes. Think of tokens as chunks of words, parts of a sentence, or data points. A higher number of tokens means the model can handle longer and more complex tasks, while fewer tokens suit quicker, simpler tasks. For example, if you need to work with large data sets or generate long, detailed outputs, a model with a higher token count—like Gemini 1.5 Pro—would be ideal. On the other hand, for faster, real-time responses in customer service, a lower token count in Gemini 1.5 Flash would work perfectly.
What’s Important to Know About Each Model?
Gemini 1.5 Flash
This model is all about speed. It’s great for tasks that need quick responses, like customer service chatbots or real-time data analysis. Think of it as a fast, reliable worker who delivers results without much delay. If you’re dealing with simple tasks that need to be done right away, this model is a great fit.
Gemini 1.5 Flash-8B
This is the more advanced version of the Flash model. It’s still fast, but with 8 billion parameters, it can handle more complex tasks. If your work involves multitasking or analyzing both text and images at the same time (like in voice assistants or image recognition), this model provides a good balance of speed and complexity.
Gemini 1.5 Pro
For big, enterprise-level tasks, the Pro model is your go-to. It’s designed for companies that need high performance and detailed, accurate responses. Whether you’re working with large datasets, analyzing healthcare information, or generating financial reports, this model offers the power and accuracy needed for critical tasks.
A Glimpse Into Google AI Studio
Google AI Studio is a platform designed to make it easy for you to experiment with these Gemini models. It’s like a testing ground where you can quickly try out different AI models by entering prompts and seeing how the models respond. The interface lets you choose from different models, view the token count, and adjust advanced settings before running your prompts. Once you find something that works, you can even export the code and integrate it into your applications. It’s a great way to see how the different Gemini models perform, helping you pick the right one for your project.
How to Choose the Right Model
When choosing the best Gemini AI model, it really comes down to what you need it to do. Here’s a quick guide to help you decide:
Need something fast for simpler tasks? Go for Gemini 1.5 Flash. It’s fast and effective for everyday tasks that require quick, real-time responses.
Want speed but need to handle more complexity? The Gemini 1.5 Flash-8B gives you the speed of Flash with the ability to process more complicated inputs, making it great for advanced chatbots or apps that involve both text and images.
Working with large-scale, high-accuracy projects? Choose the Gemini 1.5 Pro. This model is perfect for enterprises, handling big data, financial analysis, or in-depth healthcare tasks with ease.
Making the Most of Gemini AI Models
Now that you know the basics, using Gemini AI models is easy. You can experiment with them on platforms like Google AI Studio to get a feel for how they work and see which one suits your needs best. Whether you’re automating customer support, building an AI assistant, or diving into detailed data analysis, there’s a Gemini model ready to help.
Small to mid-size businesses receive PDF invoices from vendors and need to store details like invoice number, date of invoice, total amount, and vendor name. Doing this manually for all your vendors is a tedious and time-consuming process. In this blog we are going to be automating the process of extracting details from PDFs and populating the Google Sheet using the Gemini 1.5 Flash model and Google Apps Script.
The Problem with Manual Invoice Processing
Manually processing invoices is a tedious and inefficient task. It involves poring over countless PDF documents, meticulously extracting data, and entering it into spreadsheets or databases. This process is not only time-consuming but also prone to human errors, which can lead to costly consequences.
Imagine a scenario where your accounts payable team spends hours each week manually processing invoices. This not only delays payments but also increases the risk of errors, potentially damaging relationships with your vendors. Moreover, manual data entry ties up your team’s time, preventing them from focusing on more strategic tasks that drive business growth.
What can this Automation do?
This function is designed to automate the process of manual data entry inside of Google Sheets. This function uses the Gemini 1.5 Flash model to extract required/ specific details from the Google Sheet and populate the Google Sheet. Here’s a high-level overview of how the solution works:
PDF Conversion: The Google Apps Script converts your PDF invoices into a format that Gemini 1.5 Flash can easily understand.
Data Extraction: The script interacts with Gemini 1.5 Flash, providing it with the converted invoice data and instructions on what information to extract.
Spreadsheet Update: The extracted data is seamlessly populated into a Google Sheet, ready for further processing or analysis.
Now that we have understood the problem and the possible solution, let’s get coding.
Sample Google Sheet
The Google Sheet that I will be using for this blog contains the following details:
Write the Automation Script
While you are in the Google Sheet, let’s open up the Script Editor to write some Google Apps Script. To open the Script Editor, follow these steps:
Click on Extensions and open Apps Script:
This brings up the Script Editor:
Now, enter the code:
function processPdfToSheet() {
var archiveFolderId = "YOUR ARCHIVE FOLDER ID";
const folderId1 = "YOUR FOLDER ID";
var folder = DriveApp.getFolderById(folderId1);
var files = folder.getFiles();
while (files.hasNext()) {
var file = files.next();
if (file.getMimeType() === MimeType.PDF) { // Filter PDF files
var fileId = file.getId();
var pdfContent = convertPdfToGoogleDoc(fileId, folder);
var responseData = sendToGemini(pdfContent);
var details = extractFields(responseData);
// Update Google Sheet with extracted details
updateSheet(details);
// Move the original PDF and the converted Google Doc to the archive folder
var archiveFolder = DriveApp.getFolderById(archiveFolderId);
moveFileToArchive(file, archiveFolder);
}
}
}
The convertPdfToGoogleDoc() function processes a PDF file by retrieving its content, creating a new Google Doc with extracted text, and then deleting the converted Google Doc to avoid clutter. We use the Drive API and DocumentApp to perform these operations, ensuring accurate text extraction.
We then return the extracted text content, which can be further processed or used in applications.
function sendToGemini(pdfData) {
const GEMINI_KEY = 'YOUR_GEMINI_KEY';
const GEMINI_ENDPOINT = `https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:generateContent?key=${GEMINI_KEY}`;
var headers = {
"Content-Type": "application/json",
"Accept": "application/json"
};
var requestBody = {
"contents": [
{
"parts": [
{
"text": `extract the following details: Vendor Name: Invoice Number: Amount Due: Due Date: Description Tax: \n${pdfData}`
}
]
}
]
};
var options = {
"method": "POST",
"headers": headers,
"payload": JSON.stringify(requestBody)
};
try {
var response = UrlFetchApp.fetch(GEMINI_ENDPOINT, options);
var datanew = JSON.parse(response.getContentText());
return datanew;
} catch (error) {
Logger.log('Error calling Gemini API: ' + error);
return null;
}
}
We then use the sendToGemini() function to interact with Gemini 1.5 Flash to process and extract data from the text content. We construct a request that contains our prompt that specifies the details we want to extract from the content and then send the request to Gemini 1.5 Flash.
In case of errors, it logs an error message and returns null. If successful, it returns the extracted details that we received from Gemini 1.5 Flash.
function extractFields(datanew) {
if (!datanew || !datanew.candidates || !datanew.candidates.length) {
Logger.log('No valid data returned from Gemini.');
return {};
}
var textContent = datanew.candidates[0].content.parts[0].text;
textContent = textContent.replace(/- /g, '').trim();
var lines = textContent.split('\n');
var details = {};
lines.forEach(function (line) {
var parts = line.split(':');
if (parts.length === 2) {
var key = parts[0].replace(/\*\*/g, '').trim();
var value = parts[1].replace(/\*\*/g, '').trim();
details[key] = value;
}
});
return details;
}
We use the extractFields() function to process the response from Gemini 1.5 Flash to extract relevant details. We check the validity of the response, extract the text content, split it into lines and iterate through each line to identify key-value pairs.
These extracted details, such as Vendor Name, Invoice Number, and others, are returned in a structured object for further use.
function updateSheet(details) {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Invoices");
var range = sheet.getDataRange();
var values = range.getValues();
var vendorName = details['Vendor Name'];
var nameFound = false;
var currentDate = Utilities.formatDate(new Date(), Session.getScriptTimeZone(), 'MM/dd/yy');
var formattedDateTime = Utilities.formatDate(new Date(), Session.getScriptTimeZone(), "yyyy-MM-dd HH:mm:ss");
for (var i = 1; i < values.length; i++) {
if (values[i][2].toLowerCase() === vendorName.toLowerCase()) { // Compare by Vendor Name
nameFound = true;
sheet.getRange(i + 1, 1).setValue(details['Invoice Number']); // Column A
sheet.getRange(i + 1, 6).setValue(details['Amount Due']); // Column F
sheet.getRange(i + 1, 7).setValue(details['Due Date']); // Column G
sheet.getRange(i + 1, 9).setValue("Last updated at: " + formattedDateTime); // Column I
Logger.log("Updated Row " + (i + 1));
break;
}
}
if (!nameFound) {
Logger.log("Vendor not found: " + vendorName);
var newRow = values.length + 1;
sheet.getRange(newRow, 1).setValue(details['Invoice Number']); // Column A
sheet.getRange(newRow, 3).setValue(vendorName); // Column C
sheet.getRange(newRow, 4).setValue(details['Description']); // Column D
sheet.getRange(newRow, 5).setValue(vendorName); // Column E
sheet.getRange(newRow, 6).setValue(details['Amount Due']); // Column F
sheet.getRange(newRow, 7).setValue(details['Due Date']); // Column G
sheet.getRange(i + 1, 9).setValue("Last updated at: " + formattedDateTime); // Column I
Logger.log("New Row Added");
}
}
The updateSheet() function is responsible for managing entries within the Google Sheet. It first checks if the client or vendor already exists in the sheet. If a matching entry is found, the function updates the existing record with the new details. However, if no existing record is found, a new entry is created to store the information.
function moveFileToArchive(file, archiveFolder) {
file.moveTo(archiveFolder);
}
The moveFileToArchive() function is a simple help function that takes the file and moves it to the archive folder after the data extraction and the Google Sheet is updated.
Our code is complete and good to go.
Check the Output
It’s time to see if the code is able to access the invoices, extract details and update the Google Sheet successfully.
Conclusion
In today’s digital age, automation is key to staying competitive. By harnessing the power of Gemini 1.5 Flash and Google Apps Script, you can transform your invoice processing workflow, saving time, reducing errors, and empowering your team to focus on what truly matters.
We encourage you to explore the possibilities of this solution and discover how it can revolutionize your data extraction processes. Remember, automation is not just about efficiency; it’s about unlocking new opportunities for growth and innovation.
VICTORIA, BC, September 10, 2024 – Premier Cloud is excited to announce the promotion of Jason Murray-Rosel to Head of Client Success. Previously serving as Client Success Manager, Jason will now lead the Client Success team, focused on helping clients unlock the full potential of Google Cloud.
In his new role, Jason will deliver strategies to ensure clients optimize their Google Cloud investments, enhancing scalability, productivity, and overall business performance. With a proven track record of driving results and ensuring customer satisfaction, Jason has been pivotal as the primary point of contact for clients. Under his leadership, the client success team will continue to strengthen Premier Cloud’s client-centric approach, further maximizing the value clients gain from our cloud services and solutions.
Jason’s promotion is a reflection of his dedication, expertise, and ability to guide clients through their cloud journeys. As Premier Cloud continues to grow and expand its services, Jason’s role will be crucial in ensuring that clients not only adopt Google Cloud effectively but also use it to its fullest potential to drive growth and efficiency.
Victoria, BC, August 22, 2024 — Premier Cloud Inc. is proud to announce its newly established partnership with Lumin, a cloud-based e-Signature & document management solution, to elevate the efficiency and functionality of Google Workspace for its clients.
Lumin, known for its seamless integration with Google Workspace, offers a robust suite of tools designed to streamline document workflows. These tools enable real-time collaboration, secure storage, and advanced document editing directly within the Google ecosystem.
Premier Cloud, a leading provider of Google Cloud services, recognizes the importance of meeting the evolving needs of modern businesses. With this partnership, Premier Cloud reinforces its commitment to delivering comprehensive cloud solutions that enhance business operations across various sectors.
Key features of Lumin’s integration with Google Workspace include:
Efficient eSignature management
Real-time collaboration for improved workflow
Merging of multiple documents into a single PDF
Direct text editing within PDF files
Scanning capabilities that ensure documents are searchable and readable
This partnership underscores Premier Cloud’s dedication to innovation and customer success by providing tools that drive productivity and simplify document management.
For more information, please contact: Alex Shahbazfar alex.s@premiercloud.com 888-866-2230
VICTORIA, BC. August 7, 2024 — Premier Cloud Inc. is thrilled to announce its 8th-place ranking on CRN’s 2024 Fast Growth 150 list. This accomplishment affirms Premier Cloud as a leading cloud solutions provider in North America, highlighted by an impressive two-year growth rate of 242%.
The CRN Fast Growth 150 list recognizes the fastest-growing technology solution providers in North America. Rankings are based on the two-year growth rate of companies with gross annual sales of at least $1 million. Our notable achievements in Data Analytics, Infrastructure Services, and Cloud Migrations have been pivotal in driving our success and earning this recognition.
Premier Cloud’s business revolves around delivering bespoke solutions, tailored to address the unique business challenges and needs of our clients. Our client-centric approach is at the heart of our growth strategy, ensuring we consistently meet and exceed our clients’ expectations with each interaction. The company’s focus on client satisfaction is reflected in its exceptional client churn rate of less than 1%.
Looking forward, we are excited to continue our journey of growth and advancement while further strengthening our relationships with clients and partners. We remain committed to delivering top-notch solutions that drive business success and transformation.
For more information, please contact: Alex Shahbazfar alex.s@premiercloud.com 888-866-2230
The recent Google I/O conference saw Vertex AI, Google Cloud’s platform for building and deploying AI solutions, take center stage with a series of exciting announcements. These advancements empower developers and businesses to leverage the latest AI innovations, streamlining development processes and unlocking the true potential of AI.
Supercharged Development with Powerful New Models
Vertex AI welcomes a new member to its impressive lineup of large language models (LLMs): Gemini 1.5 Flash. This powerhouse boasts a 1-million-token context window, enabling applications like chatbots and virtual assistants to maintain a comprehensive understanding of conversation history. Imagine a virtual assistant that remembers every detail you’ve discussed, leading to more natural and personalized interactions.
But power doesn’t have to come at the expense of efficiency. Gemini 1.5 Flash is specifically optimized for real-time applications, offering lightning-fast response times and smooth user experiences. This makes it ideal for situations like chat support or integrating AI into dynamic web applications.
Beyond text-based interactions, Vertex AI introduces PaliGemma, a groundbreaking open-source visual-language model. PaliGemma excels at understanding and describing visual content, making it a powerful tool for tasks like image captioning, product image classification, and content moderation. E-commerce platforms can leverage PaliGemma to automatically generate engaging captions for product images, enhancing user experience and product discoverability.
Streamlining Workflows and Reducing Costs
Vertex AI goes beyond just providing powerful models. It also offers a suite of features designed to streamline development workflows and reduce costs. Here are some of the key highlights:
Context Caching: Managing the vast amount of data processed by models like Gemini 1.5 Flash can be expensive. Context caching tackles this challenge by intelligently storing and reusing relevant context data, significantly reducing processing costs for tasks requiring long context windows.
Controlled Generation: Fine-tuning AI outputs is crucial for real-world applications. Controlled generation allows developers to guide the model’s output by specifying desired attributes. This ensures the generated content, whether text or image captions, aligns perfectly with your needs.
Batch API: Speed up large-scale operations with the new batch API. This feature lets you submit multiple requests simultaneously, significantly improving processing efficiency for tasks involving large datasets.
Powering the Future with Cutting-edge Hardware
The foundation for all these advancements lies in Google’s cutting-edge hardware, specifically the 6th generation of Tensor Processing Units (TPUs) codenamed Trillium. This powerhouse boasts a staggering 4x performance boost over its predecessor. This translates to faster training times for your Vertex AI models, more efficient inference during deployment, and the ability to handle even more complex models in the future.
Think of Trillium as the engine that propels Vertex AI forward. With its unparalleled performance, Trillium underpins the development of next-generation AI models within the Vertex AI platform, paving the way for groundbreaking applications across various industries.
Conclusion
The announcements at Google I/O solidify Vertex AI’s position as a leading platform for building and deploying powerful AI solutions. With a growing library of models like Gemini 1.5 Flash and PaliGemma, coupled with development-friendly features and the raw power of Trillium, Vertex AI empowers businesses to unlock the full potential of AI and deliver exceptional customer experiences.