I find myself in a sort of twilight zone, both physically and figuratively. It’s 2 AM at Newark airport, and I’m in that space that could be America or anywhere else. As I wait for my connecting flight to San Francisco for the AI Engineer World Fair, I’ve just finished what my breakfast, after all, it’s morning hours in France.

I’m looking forward to this week. #AIEngineerWorldFair

AI Engineering is a bit undefined, it’s still not clear what the world thinks an AI Engineer is! So I’m writing down my pre-conference thoughts. It will be interesting to compare post conference thoughts.

From the broad view of the field, I see 4 distinct not mutually exclusive, versions of AI Engineering roles.

Let’s explore these:

Software Engineer that uses AI

The former aligns to something we could call Intention Driven Development. A qualified engineer works with a qualified product manager (or directly with a client, or similar) to share the design and characteristics of a system alongside the “requirements”. I’m not going to go into how LLMs can help with requirements, other than to say I am certain they can help and to do it well isn’t without complex problems to solve. The tool chain becomes more and more automated (like DevOps did for Infra).

Key point: The engineer possesses the expertise to select the most appropriate “copilot” for each specific engineering task.

While this might sound straightforward, the rapidly expanding capabilities of AI copilots introduce new layers of complexity. It is very unlikely that we will get “one Copilot to rule them all”. Interoperability becomes very important as we move into this new phase of automated development.

Prompt Engineer

Key point: The engineer knows the “best” choice of prompt techniques based on the models available.

This includes:

  • Base Model to choose
  • If fine tuning is needed, how to fine tune it
  • How best to prompt it
  • Prompts for programatic interaction with LLMs

AI System Engineer

The latter aligns to the skills someone needs to work on systems that leverage AI models (particularly Generative Pre-Trained LLMs).

Key point: The engineer knows the “best” choice of Generative AI toolset to add business value.

This includes:

  • Base Model to choose
  • If fine tuning is needed, how to fine tune it
  • How best to prompt it
  • How to manage the prompts
  • Model hosting and discovery (Service Discovery for Model endpoints)
  • How to convince people not to use RAG
  • When to accept RAG is helpful
  • Tool usage
  • Chaining of request and responses
  • Observability
  • Security

AI Knowledge Engineer

Key point: The engineer knows the “best” choice of data mechanism for each unique AI application.

Choices include:

  • Knowledge Graphs: For mapping complex relationships and hierarchies
  • Vector Stores: Perfect for similarity searches and semantic understanding
  • Chunking: Breaking down large datasets into digestible pieces for AI consumption
  • Hierarchical Small World Networks: Combining the best of hierarchical structures and small-world properties for efficient information retrieval
  • Plain old Relational data: Because sometimes, the classics are still the best
  • Not The New Kid on the Block Anymore - NoSQL: For when flexibility trumps structure
  • A mix of the above: Because in the real world, one size rarely fits all