Sujan's MidJourney Saga

How to Create a Unique Art Style for Your Serial Fiction

A small intro about myself.

Hi, I’m

Sujan Sundareswaran

, and I am writing a serialised novel exploring the question, “What if a soul does not accrue karma?”

It’s set in ancient India, and spans a timeline of 12,000 years, and heavily borrows from Hindu mythology and lore.

You can check out the newsletter here:

Zane Dickens

reached out to me to see if I could do a simple guide on how to use MidJourney better for writers and novelists, and here we are!

---

How did you zero in on a desired style before MJ? 

This was pretty simple—I started my career as a game designer, and Fable initially started as a third-person RPG script, so I knew from the beginning that I wanted a concept-art/visual development look and feel for the story.

While the images now feel fairly refined compared to a speed-paint/rough sketch, it still stays true to the digital painting roots, as opposed to 3D/anime/photorealistic styles.

But of course, I had to wait until MidJourney was capable of such styles, the initial versions produced little more than vague grotesque blobs. Here’s the first-ever image I produced for Fable using MidJourney—it’s really come a long way today.

It has no discernible features and looks more like an abstract death-metal album cover than anything else, ha.

---

A mental model of MidJourney

I think the best way to get the best out of MidJourney is to have a mental model of how MJ operates, rather than just focusing on specific workflows, as they might get outdated as features get added/removed and so on.

Having a mental model means that you have a general idea of how MJ goes about what it does, and once you have that, you might be able to predict what sort of images it would generate for a particular combination of keywords.

Firstly, while everyone calls MJ an “AI art generator”, a closer definition would be “massively complex machine learning algorithm”. This means that MJ has a vast index of various art, photography, genres, film techniques, and so on, but doesn’t truly understand what’s being asked of it. It just learns from common patterns it finds across a particular type of image, and tries to reproduce that.

For example, we can tell the difference between a 3D rendered object, and a watercolour painting, right? MJ also learns the difference between the two, by looking at a million 3D renders, and a million watercolour paintings, and learns what makes them different, and reproduces that when asked to generate an image.

The more references it has, the better its results. It probably has a lot of training data about popular trends, celebrities, well-known places, and not many about more obscure topics. This is one of the reasons I have such a hard time generating images that match the ancient-India/Dravidian-history theme, but your mileage might vary.

Ed’s Note: This is one sign of AI bias in training data, and MidJourney is finally working on including more diversity in their training data.

For example, “a biplane flying over Manhattan” produces these images—on the first try.

Whereas, a prompt like “a palm leaf manuscript about cellular biology” produces this—

A palm leaf manuscript is an ancient writing method used in India, and it looks more along the lines of—

As you can see, MJ’s result is way off, and unusable.

I had to refrain from using palm-leaf manuscript as a keyword because MJ couldn’t understand it.

So, to sum up, popular subjects get better results, and obscure subjects are a complete hit-or-miss.

---

1 — Find a style that echoes your writing

The novel I’m working on is an epic historical fantasy, set in ancient India. I needed a style that would match the theme—dark and moody—which means styles like anime/manga, cel-shading, photo-realism, abstract art, etc wouldn’t match my style of writing, nor do justice to its intricate, nuanced plot.

Think along the lines of Game of Thrones, Beowulf, Lord of the Rings, Dark Souls, and so on—that’s the style and atmosphere I’m trying to emulate. Here are some renderings.

Click on the image to see full-res.

Some renderings for characters and events for Project Fable

One of the things that MJ lets you do is emulate the style of a particular artist or illustrator, provided it’s part of the training data. Keep in mind, there are still massive ethical complications of copying artists without their consent. I’ve tried to use styles from artists from centuries ago, or in the public domain, or generic keywords such as “digital painting”, or “concept art”.

In any case, I definitely plan to redo these once I can afford a human artist, and I hope that day comes soon.

I used “oil painting”, “high-contrast”, “shadows”, and “saturation” in pretty much all my prompts. This is what gives it that digital concept art look and feel, which really gels with the theme of my novel.

I also used keywords like “late evening”, “dusk”, “dark lighting”, “night-time”, “silhouette” and so on, to drive up that contrast and stark shadows.

Once you have a set of keywords that seem to get what you have in mind, stick with them.

Ed’s Note: Check out Nick St. Pierre’s mega thread if you want to deep dive into this concept Additive Prompting. I found his structure very useful.

This is a great list—and as a pro-tip, follow Twitter threads and users as they discover more and more specifics of MJ’s behaviour.

---

2 — Find the balance of details

Let’s try a simple prompt, with a popular, well-known figure:

“A portrait photograph by Ansel Adams of Superman eating breakfast”

This is perfect. See how it’s broken down:

• Superman

• portrait

• photograph

• in the style of Ansel Adams

• breakfast

And, it knew well enough how to merge all of these keywords and parameters together to generate a cohesive, realistic image. Sure, I don’t know what the hell is in that metal cup, ha, but this is 100% usable.

This is the key to prompt engineering—keep it simple, keep it straightforward and know when to use specific details and generic details. This is something that you will develop over time, as you continue to use MJ consistently.

Let’s try a more specific prompt—

“A portrait photograph by Ansel Adams of Superman eating a custard pastry that has a cherry on top and is dripping honey”

As you can see, MJ’s completely missed the mark on all of these. all these are pretty much unusable. The only thing it was able to retain was the sepia tones of Ansel Adam’s photography, but even that I’d say is inconsistent and forced. Being overly-specific causes a loss in the cohesive style as well.

So, the first step to getting cohesive results is to find a good balance between generic and specific details in the prompt. Now let’s try a prompt that’s generic in its specific detail:

“A portrait photograph by Ansel Adams of Superman eating breakfast in a crowded bar”

This is what I mean by “specific, but generic detail”—it’s got Superman, crowded bar and breakfast right.

MidJourney is more likely to have “crowded bar” as part of its training set, than “custard pastry that has a cherry on top and is dripping honey”.

This is what gets you better results: try adding details that draw a good balance between specific and generic parameters.

On the downside, notice how the first Superman and this one looks a bit different—it’s pretty difficult, if not impossible to recreate the exact same character again and again.

Heads-up: I’m using Ansel Adams only as an example, I think Ansel Adam’s estate had an issue with MJ scraping Adams’ work.

---

3 — Iterate like a coked-up rabbit on PCP.

I cannot stress how important this one is.

As of writing this, I’ve generated ~5000 images on MidJourney, with over 90% of them just being iterations of a few concepts.

I have about ~50 images I’ve used across Project Fable, and getting the exact thing you’re looking for, can take dozens, if not hundreds of iterations. As you can see above, the above grid of images were some of the iterations I had to go through to produce this—

But, be careful of iteration with variations.

I’ve noticed that if you do variations of variations, the amount of detail is reduced drastically, and the style also loses its coherence. This is especially true of using multiple parameters, obscure keywords, and styles.

As you can see, the details slowly become just coloured blocks with each variation of a variation I try to progress with. But keep in mind, I’m using a prompt with obscure specificity, so that’s also a factor.

While this isn’t a consistent behaviour of MJ, it happens often enough to not consider as a deterministic workflow. As usual, a major factor is availability of the data in MJ’s training set.

You might have better luck with popular subjects, and not so much with obscure ones. The more obscure you go, MJ doesn’t really know what a “variation” of an already hazy subject might look like, and fails at visualising that. The further you stray, the more lost MJ becomes.

The solution is to do only to continue to do variations of the original image, so that you’re always just one step away from the original. This has given me consistent results every time.

---

Closing thoughts

A bit of philosophy and principles.

MidJourney is a tool. I really think it’s not a replacement for a real artist. Just as how the majority of Substack seems to be against AI-generated novels, the same applies to artists and AI-generated artwork.

I think it’s a great way to get your writing off the ground with some interesting artwork and supporting imagery, it’s perfect for mood-boarding and some pre-viz—but do consider hiring a human artist if you can afford it.

To me, cost is the only reason I’m using MidJourney. I cannot afford a human artist, at least not now. Once I can, I do plan to rework Fable’s artwork so that I can bring to life the nuanced details I have in mind, and the specifics that MJ got wrong.

---

Ed’s Closing Note: I hope you enjoyed our first guest Saturday Session, where we dive deep into a particular topic of modern indie and serial publishing. To give you the tools and tips to try it out yourself.

Thank you again to

Sujan Sundareswaran

for all the effort he put into revealing his process and philosophy for creating the beautiful artwork to compliment his sweeping saga.

I agree with his closing sentiments 100%, once possible I’d also love to collaborate with a human artist to truly bring my worlds to life.

Please give

Project Fable

a follow, for both the artwork and to dive into a story unlike any other. I’m a happy paid subscriber. Lastly, you can also share this post so that more people see his work.

And if you like deep dives like this served up on a Saturday, please: