It’s been a while since my last blog post. I’ve recently had some experiences that are worthy of one.
In October of last year I relaunched my SacBusiness website by dropping the old business directory and replacing it with more SEO and AI friendly sections with artwork and text describing the various categories of work that I can do. The artwork was generated by AI under my supervision. And when I say that, I am not an artist that can draw but I knew what I wanted. I found that use-case rewarding because I was able to iterate the graphics generation until I got exactly what I wanted out of Google’s Gemini.
I wanted to follow that up with an AI generated with an AI generated animated video with narration. Since I already had artwork and category descriptions, I thought that it would be easy work. Not no much.
The first roadblock that I ran into was the duration period. My goal was to create a brief video. Something in the thirty second to a minute range. That could be placed on my homepage, posted to YouTube independently or as an ad. Gemini would only do eight seconds at a time. And a video wouldn’t always get generated every time. Often nothing happened at all!
So I looked for a different platform and found the imagine.art website. It seemed like a cool idea – a single platform that has access to variety of AI generation services. But after buying some “credits” for generation of videos, I found myself in a similar boat. The duration limitation was still a thing. And what was worse, I couldn’t seem to get my artwork uploaded. Instead, I had to do everything with prompts.
Not that it wasn’t fun at first, When given the chance, AI made all kinds of changes that I wasn’t expecting: converting the drawings into people or people with cartoon heads. There was other AI slop as well: scenes with strange artwork on the wall, computers with random looking characters that made nonsense.
Faced with the ten second duration limit, I rewrote my script into scenes that could flow from one to the other. Except there was no continuity from scene to scene. Eventually I gave up on imagine.art
Which took me back to Gemini. Using my brief scene concept, I was able to create an opening scene that represented what I wanted using the art that I had uploaded and followed my script. I then generated the second scene. That went well too except for some reason Gemini inserted a subtile difference between the two scenes: the man in the yellow sweater gained a shirt collar. I went to regenerate it with firmer instructions. Gemini said no. I fought it for a while and eventually figured out that I had exceeded by video production limit. It’s aways something!
So the next day I decided that the best option was to pay for the next tier that would allow me to generate more and long videos. Except when I went into Gemini, now it refuses to do that at all! Instead, there’s a new product called Google Flow. So now I have yet another platform to try and learn. However, this one seems to be the one I wanted all along. I can upload “characters” and give them specific voices. There’s a place where I can describe their personality traits. And there’s an interactive AI prompt area where I can interact with it to get my work done.
The first pass was reminiscent of my earlier experiences with imagine.art. I gave it my artwork, my script and a go. It made up a realistic looking man called Marcus in a lifelike scene. Nice but not what I wanted. I had to give it specific instructions to do an animated video in the style of my art. The next wave was better. The AI broke down my script into smaller ten second scenes (sound familiar?) but this time with the promise of continuity between them. But as I went along, nearly every scene had to be regenerated at least once due to various corrections on their part. Specifically, the man in yellow keeps getting collars sprouting up around his neck at random. It would be funny if it wasn’t costing me credits!
Finally I got into a rhythm of generating ten second chunks, approving or making corrections and then moving forward to the next scene. Then I hit my next roadblock: the AI interface needs final approval for rendering each scene. At scene seven, the AI agent lost the approval dialog box. It just couldn’t display it any longer. But without approval, it wouldn’t do the rendering. So three quarters of the way in, I was stuck! I tried negotiating with the agent but it never could move forward. So I slept on it.
The next day, I decided to play with the agent settings. I had recalled seeing an option when I was first starting. The option was for auto-approval of video generation. I turned that on, told Flow to start and it went! So I was no longer stuck. However, the AI keeps adding or changing what Marcus is wearing. The yellow T-shirt becomes a sweater or a shirt with a collar or even a sweater with a white undershirt. And I’m paying for each of these iterations.
After two days of effort, I was finally about to get eight ten second videos. However, I didn’t know how the scenes were going to be blended together. I presumed that there was some reconciliation tool that did that. Wrong! Once I asked, Flow gleefully informed me that I had discovered a problem that plagues everyone who creates videos: continuity. However, the solution was to create snapshots of the end of each scene as the starting point for the next. Which was kinda obvious but no one bothered to tell me!
Fortunately I had some experience regenerating videos at this point. So rebuilding using the end of the previous scene as a starting point was fairly quick work. The next step was to stitch the scenes together. Flow has a tool for that. However, when I clicked the Download button, nothing happened! I finally switched from my Safari browser to Chrome and it worked. That violates all the principals of the open web and I hate that. But at least I have a video for my SacBusiness homepage now. Take a look!