Transcription provided by Huntsville AI Transcribe
So welcome to Huntsville AI. We’ve got several new faces, hence the name tags.
There’s actually four that I think you two may be new.
And Dean and Willie, y’all are new.
So if Joe, I don’t think you’ve been here before either.
Okay.
So if you don’t mind. After we kind of like do all this kind of stuff, don’t be afraid to say hi to anybody. You know, we got too many around these days to just go one by one and who does what. So after we kind of hang out a little bit and feel free, jump in conversations, meet somebody you don’t know.
You know, we’re all introverts here.
So thank you.
A little bit about Hustle AI.
Basically, our focus is to make sure that artificial intelligence is available, approachable to anybody that wants to learn about it or play with it or whatever.
So this group is we normally try to stay away from heavy corporate sponsorship or anything.
This is just us getting together to talk about stuff. So also, please don’t share your secret. We don’t want it. It’s probably not that secret anyway, if you Google yourself. So there’s that. And then I don’t have to tell you how to sign up because y’all made it here already.
Some ongoing stuff. We’re part of this A.I. Huntsville Task Force. It’s a citywide effort trying to see what types of policy changes we may need to make, what kind of workforce changes we may need to work through, things like that. We are We were doing, we were planning on doing a community wide session for like an intro to AI that Deloitte had put together. That got shifted from April to May. So be looking forward to that. It’ll be more of a user focused or as a general member of the general public. I hear about AI, hear a lot of stuff. What does it mean to me?
What do I do? What do I watch out for?
How’s it useful?
Things like that.
We get into that some at some of our sessions here, but typically these meetups are geared more towards the people with hands-on doing work in different kinds of AI. So there’s that.
Also, there’s an American Planning Association.
We got asked to go present some AI material for them. That’s on the 25th. David, I did get approval, so anybody we have that wants to go, I’ve got you and Tom posted. There’s a registration thing they can do. So anybody else that wants to go to that, it’s from 10 to noon.
Did you say for the food options to Venmo or whatever?
Yeah, it’s an interesting thing. They’ve got like a Google registration form kind of thing. You fill it out and it lets you pick food options from Jimmy John’s. And so I got through that. I got to the bottom. It’s like, well, here’s my Venmo info.
So it’s literally somebody doing it.
I mean, that’s how they roll.
So, yeah, if you want that, that’s available.
It’s at the new City Hall, so if you haven’t been there, that’s also an interesting, a good way to get in and see what the place looks like. It’s pretty nice. Let’s see, there’s another piece that he dropped I wanted to share because this was Scott Ross, if you don’t notice, is from Hudson Alpha.
Talking about Alabama Launchpad. So if you are interested in startups or funding, things like that, this is a pretty good way to meet other people that are interested in the same types of things. That’s another thing on the Huntsville AI side. We’re not, this group is not a business incubator. I’m not all that great at it.
I’m okay at AI, but there are plenty of groups in town that are more focused on how to start a startup or, you know, things in that area.
So I just wanted to mention that that’s May the 13th and it’s going to be here. So you know how to find it. So with that, we’ll jump into, wow, that’s fairly horrible.
agent platforms.
So we did a session, I don’t know if it’s a month ago, just general AI agents. There’s a free course from a company called Hugging Face that walks you through a bunch of different types of agents, a bunch of different types of tools, things like that. So their definition is that an agent is a system that leverages an AI model to interact with the environment in order to achieve a user-defined objective.
It does reasoning, planning, and executes actions often through external tools.
So that’s basically their definition of an agent.
My definition of an agent is a model or something that does something for me that I didn’t want to do myself. Call and make an appointment or schedule a service appointment with an auto dealership or find available hotels within my budget by looking at my calendar or on this weekend. You know, something like that.
Those are the types of things that are easily available.
are possible now. Something I’ve been thinking about actually building out is something along the lines of, hey, go look at the internet and find the top three topics being discussed about AI.
So a search thing. Write some sample social media content, including images. That’d be like the next step. You may have a third step for review for relevance and make sure it doesn’t have anything offensive in it, you know, because you really want to do that. And then possibly email me the results to this, you know, something like that.
So those kinds of things you can do.
It is kind of interesting.
It feels a lot more like I’m just using an automation platform that’s just smarter than it used to be. Because most of that stuff you could do with Zapier or if then, then that, or some of these other systems for, I mean, Microsoft has a boatload of stuff in their business operations type stuff where you got incoming emails and this looks like an invoice. Let me ship it over to this folder and scan it for this price, notify this person. And if this person doesn’t approve, fall over to the secondary approval. I mean, all of that, just automation systems.
These seem like they’re the ability to make them a lot smarter. So that’s kind of where we’re at.
I’ve got a couple of opinions after going through a lot of this up front. We looked across several different vendors and platforms and things that provide either libraries that let you build agents or systems that host agents or things like that.
And there’s no clear winner that I could find.
You’ve got, you know, some of the ones we’re going to go through were specifically from the Hugging Face, you know, tutorial set.
But they all have like their one unique kind of thing that they do well.
Some of them actually incorporate others. So you’ll be in the middle of one and all of a sudden you say, well, you can also use the tools that these things built over in their platform. Like, great, why am I not over in their platform, you know?
So there’s a lot of just back and forth, and it seems like a mixture of, you know, what you get, what you find.
It’s nearly kind of like pick your own journey or a buffet of all of these different things that you can look at. The only one that seems to be different in that case is the model context protocol, which is probably because it’s mostly just a protocol for serving tools.
It doesn’t really do much more than that, which I really appreciate.
I love it when something does the one thing that it’s supposed to do and it doesn’t try to do all the other things. And then the other piece, back on the line of, you know, if the service is free, then you probably are the product, that kind of a mindset. A lot of these exist.
Some of them appear to exist just to get you to use their particular brand of inference.
And with that one, I’m talking about Hugging Face. You go through their tutorials and you turn around and everything you’ve got calls back to their provider for doing a model or serving up a model along. And they’ve got your key to be able to do that and then turn around and charge you money later. if you actually try to go to production with that. That said, it is a really good way to build a tool that uses a large selection of models, which is, that’s something else that’s a little different.
You’ve got, in the core of any of these agents, you’re going to have an LLM or two or three, depending on what you’re doing.
And that’s typically where your big money is going to go.
as far as the cost of serving up an agent like this. So these models that do the reasoning of step-by-step, it’s kind of like similar to what you’d find with a chat GPT, but you’d say, hey, I want to plan a trip to Chattanooga.
Here are my constraints.
Think step-by-step and do this kind of thing. And actually what it will do is it’ll look at your thing. It’ll look at what you tell it. It’ll go through and say, oh, I see you have a group size of four.
At that point, it knows if it’s going to make reservations at a restaurant somewhere, you’ve got four people. You could put things in like how far you can drive before you need to stop. You could put dietary restrictions, things like that. But the more complex that you make this thing, the larger… model you’re going to have to have to be able to deal with it. And the larger model you have, the more you’re going to pay to use the model. And none of these, the other thing I bring up is most of these, if you are just trying to build something and play with it, you’re not even really in the tens of dollars.
You are, I mean, really, really low price just to get something up and go play with it. I would suggest doing it. A lot of these models have free tiers that you can use and you’re not likely to move too far out of that, if at all, to go play with it and try to do some things.
So the other reason that I’ve seen some of these exist is As these models and these agents get more interesting, especially the ones that deal with possibly sensitive information, the ones that deal with money, the ones that would have some kind of a negative impact somewhere, you have to have some way to tell if it went wrong, why did it go wrong?
a.k.a. who is at fault, if you will.
So there’s a lot of tracing type things that have been kind of put out on the market to help trace through this model decided to do this.
Here was the inputs and here was what it decided, you know, so that you can go back later and figure out first, why did it go that route?
And then second, if this is a security issue or whatever.
what other tools do I need to put in place?
So there’s two, especially Langraph and Langsmith.
Well, tracing tools were Langsmith and LamaTrace.
Some of these platforms get you to use their tools and of course you use their tracing things and then you try to deploy this and you find out later that, oh, that tracing has a cost associated with it.
Oh. I didn’t know that when I first started and it was similar to my MailChimp email earlier. Everything’s great until you move out of toy land into production land. So understanding where some of those costs are and why they’re putting this stuff out. The good news is there are some really, really good tutorials out there to go from start to finish on building some of these types of AI agents and even deploying them. And there’s even some third-party companies. Weights and biases have been around for a pretty good while.
We were using them, this was pre-COVID, I think.
I do feel weird that COVID is now where I like draw, I’ve graduated from college, got this job.
I mean, it’s like one of those, everybody bases time from 2020 now.
Anyway, so there are some third-party companies like that that are out there with their own plugins to help you trace your own type of thing.
So let’s see.
Any questions so far on that?
Just what an AI agent is or kind of what they do?
First, we’re going to jump into Hugging Face, which is, this is the tutorial that we walked through the other day on using their framework.
Most of what their pieces are geared towards is how to build your own tools.
And then you serve your tool up with this agent and it knows how to use your tool to go do something. So let’s say that you had a very specific data center kind of a thing set up. You had all your documents set up in these kinds of directories and whatnot, and you needed to provide the information and those documents to an agent to go answer questions on your website, you know, from a user-facing perspective. This would let you build your own custom tool based on how your data is set up. Maybe you need some way to filter out certain things. You can put that in your own custom tool. The interesting part about the hugging face that’s a little different from most of the others, the way they approach it is you define a tool to do something. And you put your big text description on here is what this tool does. Here’s how you’re supposed to use it.
Here are the inputs it takes. Here’s the response it gives back and all of that in one big text block.
And then you write your tool and then you give it to the model and you hope the model makes the right choice and uses your tool based on how well you describe it. And so I feel like my wife’s a preschool teacher and I can’t tell you how many times she has told me to make good choices. I feel like we’re offering up these tools to a model and say, please make good choices. Please use my tool when you go to my directory structure instead of searching the web for it, you know, because I’ve got data here.
It’s not on the web.
So you have to write prompts in certain ways to make sure that it goes to your side first or that you’re asking for the right thing. So we’ll look into some of kind of what they provide. Their whole thing, Licensed Call, is kind of built around 007 and Alfred.
Their whole tutorial, they’re trying to tell the story, keep it interesting, so it’s okay.
They got, let me see, conceptual guys, tutorials. Just to show the kind of level of goodness.
on some of this stuff. They have a really good website tutorial set up where it’s easy to walk through. You can bookmark things and come back later.
I’ve gone through their first like four units of their agents course usually while on my cell phone while we were on our way to Georgia while my wife was driving.
You know what I mean?
It’s really easy to, you know, it’s… it’s pretty easy to pick up and come back and keep track of where you’re at.
So their whole basis is you can import some of their agents.
They’ve got some that are, they call it code agent, which is their hugging faces proposition is that if you write your tool in actual code, like Python code, that it can actually take that along with your description and do the thing that the code is saying to do, which kind of makes sense.
You have to be pretty precise with the language you use and the syntax you use to actually have a compiled or interpreted language actually do something.
Other than just using some places or some methods for tools allow you to define the action in just basic text.
Things like look in the directory named taxes.
It’s almost like you would be trying to explain to the computer text what to do. And there’s synopsis.
supposition, I guess, is that it’s much better if you actually write all that out of code because you have to be precise and it makes them easier to parse and such.
So with their approach is you define the tools, you build up the model, you give the tools to the model.
Well, actually, the tools go to the agent, the model goes to the agent, and then the agent actually calls the model with your prompt or whatever, and it’s smart enough to go use the tools that you provided. to accomplish what you’re trying to do. The other thing, let me see if I can get further down here. And they’ve got a couple of existing tools that you can use.
You can actually also have it trace some of the logs as it’s going through.
Which tool are you calling?
What are you giving that tool?
things like that.
Multi-agents, and this is where I got super confused on some things.
They treat the multiple agents, which is where if I’ve got several, let’s say I’ve got an office staff and I’ve got one person I need to go make reservations, I have another person in charge of billing. I need all of these people to work together. It’d be similar if I had put those into kind of AI agents where I’ve got this, they need to at least interact with each other and pass information from one to the other.
So if I go, let’s see, reserve a rental car somewhere with one agent, I need the information for that to get back over to my finance agent.
To know that, hey, we got this thing coming up and you should expect to charge or whatever.
I don’t know if I’m pretty sure there’s some systems that are up at that level at this point where there’s actual money transferring and things happening.
I’m not at a point where I would put my token in to let something. Not only think of a hotel booking based on your calendar, but along with the information to charge your credit card for that booking. I’m not quite at that stage yet. I’m still in the, yes, go find it, but then check with me.
There are probably other systems that have a… another kind of an agent that goes back and checks the work of the first one to make sure that that’s, you know, that you can’t make decisions by yourself kind of thing.
The way they do their multiple agents is similar to the way that they do their code, their tools.
So not only do I have tools that I had to make a big description of what it does and how to use it all, I’ve got other agents that I had to make a big description of the agent and what it does, how this agent interacts. And then I take all of these and I give it up to the managing agent and I ask it to please make good choices. So we’re still it still feels really fuzzy as far as, you know, how it works.
You’re going to get some not repeatable type things, because even even if you ask the same model.
I mean, it’s almost like go ask chat GPT the same thing on a couple of different sessions or a couple of different days.
If you’re not using the same type of, I mean, there are ways to make it repeatable.
If you’re not following those mechanisms, you’re going to get slightly different answers.
They might all be right.
They might be worded different, you know, things like that. So that’s basically the hugging face approach. A couple of ways to visualize it.
We’ll leave that at the hugging face. The other one was Llama Index. Llama Index is another platform.
It seems to be mostly geared towards data ingestion.
There was one we had come out with, I think it was last year, we were going through how to build uh rag retrieval automation generation you know retrieval augmented generation uh systems uh and we come across this really nifty library that would just rip through pdf files and give you the information that’s in them which is not it sounds easy but it’s not It will really drive you nuts, especially when you get two column PDF files and especially when the lines on each column offset by a little bit and you give it to a normal library which tries to read left to right.
It reads the first part of this line and drops down the next part of this line and it thinks that’s a sentence because it’s not that smart. So there’s that. But they had built a nice PDF parser that we wound up pulling in for that.
That’s the first.
I didn’t even know it was an actual full framework. So if you’re looking for a tool that you can use to ingest data from just about any source you can think of, they’ve got things that will plug into Outlook, things that will plug into Gmail, things that will plug into Google Drive, Dropbox, about any kind of database you can imagine.
if you wanted to build a system that had reach back into a lot of different places.
So that’s kind of their bread and butter.
And of course, anytime that I’m looking at something that is open source, but then I also see the thing across the top for book a demo, it’s kind of like, Anyway, but I get it.
So they’ve actually got a lot of open source.
Most of the tools they use or provide are open source.
And you can host your own stuff. Or if you want to build something and run it in their cloud and pay them to host all of it for you, you can. which it is kind of the, that’s probably one of the differences between if we’re talking to, you know, your normal member of the community is just trying to do things and they’re not like heavy into building their own infrastructure and all that kind of stuff. This is probably a good place to go. There’s help desks and people to ask when you run into problems and things of that nature. where a lot of the times if we’re hosting something ourselves, guess who has to fix the problems when it goes down?
So there’s definitely a trade-off. Let’s see, if I drop down, that’s not where I was trying to go.
They’ve actually got a Llama Hub that I’m looking for if I can find it.
solutions, community, careers.
Let me check the other link I dropped.
Probably.
Let me go there.
Yeah, so Llama Index.
So if you are looking for something that does retrieval.
And maybe it’s in Amazon Kendra or maybe it’s a Mongo database or, you know what I mean?
It’s a, they got a large amount of pieces already kind of built up. And with their framework, they make it super easy to go pick this thing off of this project, pull it in and hook it up to this other thing from this other project and kind of build your own system. The thing I did like about what they provide is they got this concept of a workflow with a… I was trying to remember what these, if I go back to the small agents, I’m not sure if I can find this or not.
They had, anytime you’re in the hugging face part, as a lot of the work that they do in their actual wrapper code to build a tool or to build a model or something, it winds up generating some very, very peculiar and intricate prompt that then gets sent over to the LLM.
And at the very end of it is like some way for it to know when its job is completed.
And if you don’t have that in, this thing will just keep running and running and running and running and polishing that answer that it’s trying to give you for a long time and charge you a lot of tokens, if you will.
The Lama Hub, not Lama Hub, the Lama Index approach. has more of an event-driven model.
So I have a stop event.
When I hit a stop event, it stops.
I don’t have to worry about it or things like that. And I swear this felt much more like Wisp and putting some things together. So you can define your functions or your steps. So this step takes the start event and gives you a stop event when it’s done.
Single step thing.
Should do the same thing every time. They’ve even got some tools in here that will actually give you a picture of what your agent is doing. You know, I’m starting, I’m going to generate something, I stop, now I’m done. You can actually do loops and branches based on this stuff. And again, this is kind of… This is the kind of thing where we’re back into instead of trying to explain what I need a model to do in text by giving good descriptions to my tools and a solid prop to my model and hoping that it makes good choices.
In this one, I’m making the choices for it. But again, I’m back into this is like writing code. You know, this way, you know, I mean, it’s probably a different use for a different type of person. I’m still waiting on the model that actually takes one from a small agent to Hugging Face and says, based on this, build me the code that I need to put into my other framework for the, you know, for the other one.
But you can do some interesting things here.
The other concept to think about is if I’m making different calls to either different agents to go to this or an agent to go to that or a tool to go look this up or do this other thing is how do you pass information or share information from one tool to another tool that they may need to both have the same concept. So if I’ve got one agent going to book a hotel, I’m not giving that thing my credit card. I may have a whole nother agent over here that may have access to this information, but that one doesn’t. You know, so that one may go find the thing or whatever and say, hey, after approval, this thing needs to make the, you know, the reservation.
But that means I have to I have to share that information between in a controlled manner.
So I’ve got some information I want to share, some information I don’t. And how do you do that?
And that’s where some of this comes in as far as managing state.
The other fun thing is when you make an agent, and a lot of times we’ll do something just for us to play around with, which is fun. How do I make a bunch of copies of these things that are out running around and make sure they don’t? So if I’ve got an agent and Jack’s off using it to book hotel rooms. I don’t want it to use my credit card because I’ve got the same agent running over here. So you’ve got to keep clear lanes and it gets pretty interesting in how some of these things are deployed, hosted, and whatnot. So that is Lama Index. The other one we were going to cover is Lane Chain.
Oh, actually, I did want to mention the… The Lama Index one, when I think about an agent that is doing something on my behalf because I didn’t really want to do it myself, you know, you think of it like an insurance agent. You can personally go find all of the stuff the insurance agent does. They’re not actually writing you the policy. They are working with a different company that writes the policy or works with, you know, you could actually go do all of that yourself. But it’s a lot to know. It’s a lot to, you know, so we actually. have them do it for us. But there’s an action that happens with some kind of an outcome. Most of the workflows I found on Lama Index were much more along the lines of a very specific either chat bot or some other augmented rag.
So it’s like a rag plus or a re-ranking with a rag or, you know, it’s slight tweaks to things that… felt a lot more like our standard search. I want to search, but I also want to include my company’s data in my own search internally.
Okay, that’s great. I don’t know that’s what I would call an agent.
I’m still the one doing all the things.
You’re just giving me access to different tools or augmenting this with also go find any applicable market data or something, you know, I mean, you could do a lot of things like that, but so I guess it is time-saving, but it’s still, I don’t know, it didn’t quite feel like an agent to me. Of course, I might be weird.
You think it would fit more in with like the multi-agent workflows where you might have, say, something in small agents that can call out to this, use some of that information to make some of its decisions rather than having you, the analyst in the loop, have an agent in the loop?
I think so, yes.
Some of what I saw were things like almost like a team of data scientists.
One of them was actually like that, was, hey, I want to pull this in, but I’ve got a team that actually goes back and looks for specific items in this particular data set and then augments my data set with their opinions, something like that.
Another place I’ve seen that they pull in with this, is an evaluator for how well does this answer reflect the actual prompt that was asked.
I don’t know if you’ve ever asked ChatGP to PT something, and it gives you an answer, and it’s probably a good answer, but it’s nothing. It’s not what I’m looking for.
It’s not batching. How did you get this from the words I put in?
Sometimes that happens, especially if you’re looking for some obtuse thing and it can make the jump from what you asked into some other kind of a domain.
It’s like using Bing instead of Google.
Yeah. I mean, there’s actually a unique trick to how you ask certain search engines things to get the info you need.
One of the things that I’ve actually got to write up on my LinkedIn, somebody gave me a recommendation, a commendation one time for my use of Google to find obscure information. We were at the time using a framework called Eclipse to build UIs and the, I think it was Luna, was the name of their build at the time. So I was looking for Eclipse, Luna, this. At the same time, I think Luna, Eclipse was like some Star Wars character and some other things. Every time we would look for stuff, we would get Star Wars. And it’s just like, no. Yes, the words mean that in that domain, but that’s not what I’m looking for.
It’s the same kind of way even today.
I mean, if it’s smart enough to know that, hey, you might be talking about this because the word’s concept feels like this. So there’s a way to evaluate answers to make sure, first off, is it relevant?
Does it have the right stuff in it?
So a lot of these frameworks or agents kind of thing, let’s say I have a fact checker.
I may want an agent that does nothing but check the references to make sure there are actual links that can be downloaded.
If you’ve ever done that, when it gives you references, you look at it, it looks right, but you click it, and it’s not a thing. It’s fun. I can see things like that as well. I did research some stuff like that when my company was going through a tutorial.
We built this chatbot for doing customer service.
I was like, okay, well, they want to ask a question about this screwdriver or this fan I bought is not working right. Can you refer them to this page in the manual? Answered.
these 10 questions correctly and they called them judges for how it would evaluate how the actual agent was performing but you could score it get a trace of how it performed throughout that and you go through the different tunings of it figuring out how you wanted to get that so that you could get the 10 out of 10 and then that’s what you’d send out production right yeah that’s cool Did y’all use any kind of a, like a weights of ICs or anything like that? Or was Databricks the one? They like you to use everything in their sandbox. If you’re familiar with Databricks, they really want you to pay them. Yeah, they are. Yeah. They’re not the only ones.
There’s a, but they’re also a pretty, pretty beefy open source supplier.
of a lot of other frameworks that are underlying things we use and then they turn around and they make their what I don’t mind their approach as much because there’s they’ll do what a lot of people do but do it better or make it easier to use or something. We have I think I’ve been there before. It’s really bothersome when you get into something like that and it is hard to leave. because of the way that you happen to have architected your system to fit this particular thing. It’s like, oh, I really, like right now, I’ve got a transcription engine thing. We’ve got, if you hit transcribe.hsp.ai, it’s a thing. It’s all AWS. It’s all Lambdas.
It’s super duper cheap.
It doesn’t go out to anywhere else. But to do that, I had to build it using AWS-type stuff. I’m trying to figure out how do I pick this up and run it on RunPot. Oh, a whole other, you know, I’m basically back to re-architect some things. Cloud agnosticism is a whole fun topic. Oh, my gosh. Todd’s not here.
One of the things Todd has actually found, he’s having to, he was, I don’t know if he still knows this, move up to like a Joomla. uh web host you know content management system is upping their version or something i think he’s been using a lot of ai tools just to go hey how do i go from this config i need to upgrade to this other version and have it actually go advise him on how to uh you know how to reconfigure to meet the new spec it’s kind of interesting um lane chain is Maybe the most production ready thing I’ve seen so far from a, from a, hey, if you, plus they got a really slick website.
The thing at the top that changes colors as I scroll up, that just like, that’s nice. I would, I would literally lose like five hours of my life just trying to make that thing do that.
And somebody probably did at some point. Their main products are LangChain is actually the piece underneath that provides the agent platform of a way to take tools and combine them with agents to do things. LangGraph is kind of there. I haven’t had time to run it yet, but it’s more along the lines of one of these drag and drop.
on a windowing system, how I want these agents or tools to be connected, how I want them to interact, and then visualize that and then deploy that workflow out to their hosting platform.
And then LangSmith gives you, again, the pieces to basically debug, monitor, get audit logs.
of all these transactions and stuff like that. So we haven’t really, I haven’t had too much time to get into it. Let me see if I’ve got, I think I had, that’s the only impact. That’s the only one I had for the lane chain. But they’ve also got a lot of Building blocks and things that you can use, which I know we saw in the Lama Hub part where they had ingesters and things that, let’s say you were pulling in a document off of Google Drive and you need to break it into paragraphs in order to send it over to some other database or analyze it. There’s most likely at least one or more. modules or libraries in link chain that can do that.
There’s probably some in Lama Hub that can do that. There’s some in Hugging Face that can do, you know, it’s at the point where you can mix and match a lot. Some of the things I found, I think it was even in small agents. I don’t know if I may have to go see. I don’t know if it was on this one or not. But some of these tutorials you’re getting into, I’m in this Hugging Face tutorial and all of a sudden it shows me, well, here’s how to pull these tools off of Lama Hub.
I’m like, well, I thought that was a whole other thing.
Actually, you can mix and match a lot of this stuff. It’ll make your head spin because you get in the middle of it and I can’t remember where I started from and I’m back to the same point. So that is… the basics that I’ve been able to get through on these three AI agent type things. The other two I’m going to hit in just a second.
There are two unique kind of aspects of the agent stuff that I’m not quite sure have been ironed out yet. One of them is attribution. If you think of it from a marketing perspective, if I put up a billboard and people show up at my store, is it because they drove by the billboard or is it because they were walking by my store? How do I attribute that customer to that piece of marketing material? Or if I’m running ads in multiple places, how do I know which ones are actually working or not working? If you get into the attribution part, If I need to use a person’s email or something like that because I’m doing an action on their behalf, I need something that tells me that I’m authorized to act on their behalf. To then give to this other tool to say, hey, I’m going to book something. Well, to book, I need an email address.
Well, okay.
That means whoever it is on this side had to authorize their email address to be used. You might be three levels deep. So figuring out how that works, I could imagine some of these tools might have different prices on to go do certain things.
How do I charge that person differently for this thing while charging person B for the, you know, it gets really interesting.
And then the other side of the whole coin is privacy on data. How do I keep the secret stuff secret and the public stuff public? And because we’re talking about sharing data between tools, we’re talking about sharing data in a lot of places. And in the culture and domain I work, there’s lots of rules. And that’s just the DOD. You go on the medical side and there’s probably more rules. You know, it gets really interesting with uh where data comes from uh how it got there uh and such and some of those uh frameworks still aren’t quite there yet um we also have as your agents are searching for something and somebody else’s agents are the ones serving it up how do you give your agents to give permission for this agent to use the data you gave this agent yeah the additional yeah and there’s likely other unintended consequences of some things. I don’t know if you have ever looked for airfare and then gone back later to book the flight and the price is different. Huh. Even if you try like a unique browser or something, it probably still knows that somebody from this IP address was looking for a flight.
So they know if you come back, they know you’re interested in buying that flight.
You’re not just browsing at that point. There’s actually some interesting tutorials on how to get good prices on flights, on what days to actually go price versus what days to book. So you can imagine some of these agents coming through.
So now your search turns into a… So it’s kind of Google’s method. Search from a massive amount of people is… data sources for other people that need to know. In other words, if I’m trying to sell something and I’m trying to figure out what brand I should offer in my store, an easy way to know is what were people searching for on Google in the last month. Google knows that. They will provide that data for you happily.
And you can then pay for a data.
So all of this information.
Even when you’re looking for things, just knowing that people are looking for the same thing is a key information going forward for some other entity.
One of the early things that we were looking at playing around with, we figured out a way.
Anybody use Uber much?
Have you dealt with Uber searches where the price goes up because everybody’s trying to get maybe a, you know. Some kind of a ball game just finished or a concert finished. Now, all of a sudden, here’s all these people.
And so the price goes up.
So it’s great if you’re a driver and you’re in that place because you make more money. So we figured out a way that you could actually take all of this information and predict where surges were going to happen. And so if you do that and then you make that tool available to all the drivers, all the drivers go to where the surge is supposed to be. Guess what happens?
There’s no search.
All the drivers are here.
So it’s one of those, you can’t, you know, one side of the thing, even though you didn’t mean to affect the other piece, it gets really, really interesting.
So if you don’t have a lot of that comes back, I’m going to imagine a lot of these trace tools and auditing tools and things like that are going to be where a lot of this effort winds up being, more so than building these agents themselves. Another interesting one kind of along that line, and this is more for MCP than some of the other ones, but now with MCP, you have these remote servers where people are hosting the tools as well. you have this idea of, you know, someone could create a rogue tool where they’re saying instead of sending back the information you think, it’s sending back telling your model running locally to go execute, hey, remove all the files, do an rm-rf slash, right?
Or you could tell it to do all of these things where you essentially have like a malware injection from like the tool coming back as well. So now Cursor has a, in their like yellow mode, you can specify like no matter what, never run these types of instructions because you could, theoretically, as you’re calling out to these remote tools, whatever comes back, you’re just kind of trusting that that’s kosher on the way back in.
Right.
Or the thing you are, some of the stuff I’m seeing on some of these tools, actually, we’ll jump into MCP real quick.
A little short on time.
I’ll probably just stop with this diagram.
So model context protocol is a way to basically, if you have tools that you want to provide to agents to be able to use, you can spin up this server.
It really felt like old school WSDL type stuff where it’s discovery and I can put my services in here or my tools in here and I can explain what they do and then offer this up.
Or if people want to come use my tool, they’ll pass the information. My tool goes off and executes, patches things up and shoves it out. Really, really good if you need to protect certain pieces of information. uh, that may be on your system, uh, through some kind of an authorized kind of a connection. Uh, but this, um, this has security nightmare written all over it for me, but, um, you’ve also got, how do I authenticate that? I mean, almost back to the attribution piece. How do I, how do I use a token to pass along to say, you know, this is who I am.
Um, how do I verify that you are who you are?
as far as your server goes.
You can see a lot of interesting spoof attacks and things where all of a sudden it’s calling my server instead of the other one and I do the same thing that the other server does but I also track your information or whatever. So there’s a lot of interesting stuff here. You can have all of this done completely locally, though.
So if you are worried on the security side, you can get the benefits of MCP of having that like standard framework across all this stuff, but have it all entirely locally.
Right.
I do really like it.
I think I’ve seen a couple of places that take MCP and wrap it in something that provides the security layer instead of MCP itself having to do all of that, which is really nice.
And then the other thing I’ll hit real quick.
because I haven’t really had too much time to jump in because it dropped not too long ago.
Google came out with this agent to agent thing, which is a protocol that lets one agent, AI agent, go talk to another AI agent and kind of work in cop search. Yeah, their whole agent space got a big update last week, week and a half. Yeah. Oh, I haven’t had time to hit it yet, which one of the mantras, the two things that I usually go with on AI, if you’re looking to learn it or something, is just realize that half the stuff that we do was just totally made up and happened to work when they tried it.
And they spent a lot of time trying to get a proof as to why it worked. There’s that. And then the other is don’t worry about if you’re going to. produce a product that is going to get, you know, overcome by other products. You don’t worry about it because it will be. There’s no, you have zero chance of being, you know, you just have to be good at it and get something out there. Yes, this was what, two weeks ago? Something like that.
And we were like, you know, Hugging Face has this agents thing. They’re like on week number four. And all of a sudden, Google drops.
It goes, oh, yeah, we do agency. And it is interesting.
There was a place back here at the very bottom where, no.
They’re actually combining.
It looks like they’re not trying to get into the MCP space.
I don’t know where it went, but there was another piece.
There we go.
Don’t worry, MCP, we love you.
That feels like a trust me. Anyway, it may actually be genuine. So basically, you’ve got the model context protocol that says how I provide tools, and then this side, it’s a protocol that says how do agents talk to each other.
So it seems to be a fairly compatible approach, and we’ll probably dig much more into it at a later time.
I was really hoping when I started putting this thing together that there would be, okay, I got three things. I could go Lama Index, we could do Lane Chain, or we could stick my hugging face, and there would be one that would jump out and go, hey, this is probably the way to go, and it is not that. Have you looked at Pydantic AI?
It looks better to me than, it’s also newer, and that’s unfair, but Lane Chain’s been around for a while, and the biggest complaint for everybody is it was written before people knew what they were abstracting.
The number of abstractions it introduces is pretty high.
If you want to know what you’re sending to the LLM, don’t use LangChain because you’re not going to have any clue.
But Pydantic AI, if you’ve used their normal library for just data validation in Python, it’s that same group, but doing AI agents.
They were like, yeah, we thought we’d just use other frameworks, and then we found out they all sucked. We’re taking Pydantic and extending it to agent frameworks, and it was pretty new when I last looked at it, but they’ve added a bunch of the model providers now, and it looks pretty nice by now.
Is it leveraging Instructor as well inside? I don’t know. I don’t think I’ve seen Instructor in it. Okay. It’s worth a look.
Yeah, their model support, they used to not have a bunch of them, but their model support looks good now. They’ve got AI, deep seek, enthralling, Gemini, Lama, Grok. Yeah, they’ve got patterns in there for how to do simple agents and how to do multi-agents and the multi-agent sections. If you don’t need this, stop reading here.
That’s intelligent, right? The simpler stuff’s that way. Instead of, you know, get you down and lock you in and now you’re paying regardless whether you want to or not. Yeah, I didn’t see any of that here. They do integrate with some of their logging, but I think that’s open source too. I’m not sure.
It’s a fairly good pattern that usually it’s the fifth or sixth one that actually figures out what all the previous ones screwed up and how to get it.
Now we’re pretty sure we know what we’re doing.
So you get the right place. like I said, the abstraction is in the right place. Yeah. And most of the other models use Pydantic as a library anyway.
Yeah.
Because it’s the right way to do tated validation of Python. Right. So let’s stop with that.
Let’s see.
Online.
Actually, I don’t know why I’m going to look there.
I’ve got Zoom call over here. So any folks online have any questions?
Best way to get started.
Pick one and code something. Do what?
Pick one and code something.
Pick one and code something. Yeah. If you’re not planning on hosting your own, your own host and doing things like that, something like the lane chain that actually provides a hosting thing where after you build it, it’ll do it for you and spin it up and let you play with it. That might be a good approach. Didn’t you say the Hug and Face tutorial gives you a couple practice runs for free? I burned through the number of tokens I could use within an hour.
At which point, I don’t know if I’ve got that posted or not.
Hold on. I may need to go. In our Hustle AI thing, hugging face agents. Let me see if I actually updated this. Just make an agent for cycling. I might have actually updated this code agent. Yeah, for the hugging face agents, if you’re going through their course and you run out of tokens, I’ve got the same thing they do provided here except it uses their light LLM model along with your OpenAI key to go back to OpenAI and use whatever models you want to use instead of using the hugging face and French ones which again that was one of the yeah it’s their tutorial. I get it. They want you to use their stuff. I mean I appreciate the work they put into the tutorial because it’s pretty good. But then when you’re trying to go through, and I can’t execute this the third time because I’ve run out of tokens.
I feel like I’m a kid at the fair with tickets trying to ride rides, and I’m out the first five minutes. So that is posted if anybody wants to play around with that.
And also, for those that are new, everything that we do, If you go find HSV-AI on GitHub and go to your presentations, you know, we get tired of folks asking about slides, so we go ahead and do it beforehand.
And normally what we’re showing is what’s already posted online.
So that’s kind of our approach going all the way back to 2018. Any more online? I think Mike was it. Any questions in the room or any comments on stuff?
I think pedantic might be the next one to look at.
Apparently this is the year of AI agents.
That’s kind of why we’re kind of headed down that path.
The video generation stuff was really, really cool. I’m kind of torn because that was more fun. This feels more like work.