Skip to content
View AISidesKicks's full-sized avatar

Block or report AISidesKicks

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
AISidesKicks/README.md

Hi There 🤝

I spent 10+ years as QA/QE & Integration Engineer - hands-on experience delivering quality on complex, high-stakes SW projects across TELCO and RETAIL industries.

I spent 10+ years as Architect & Implementation Engineer - hands-on experience delivering complex HPC, ML and AI clusters and HW/SW solutions based on NVIDIA for FORTUNE 100 companies.

It was a very funny task to fit it all on my LinkedIn, in the age of AI we are now "Applied AI Solution Architects", not just SRE.

For many companies most interesting learned lesson from AI PoC and MVP is: you suddenly have patched together 100+ AI building elements and token consumption jumped through the roof, it doesn't scale and it doesn't have defined realistic tokenomics - so next time bring experienced Integration and Inference engineers on board too.

PLEASE! We know how to scale and measure - we have System Thinking and End2End approaches in our toolkit too. See Thinking in Systems

Why Netflix is betting on systems thinkers—not specialists—in the AI era - Elizabeth Stone CPTO | Lenny's Podcast

Watch on YouTube

I know how to do my job. AI doesn't! PERIOD!!!

Well:

As AI gets better and better at code, the AI harnesses keep improving themselves and code can be checked by itself. This builds a self-improving RSI loop that extends beyond code, into Infra and Integrations too.

But for now they do it in a naive, messy, not caring way, because they live in labs with siloed environments, not real enterprise ones - this can potentially encourage reward hacking.

AI can handle narrowly focused tasks fine, but it trips over the complexity of integrations and long-term stability needed for infra - unlike code and the predefined environments in labs, the enterprise is messy and variable all the time.

AI is, for now, very LAZY when it comes to REAL Integrations or Infra management, and it can be very dangerous without the right human proactive guidance.

YES! An Agentic Infrastructure will one day fully replace the current mostly template-based Infrastructure as Code (IaC). But for now, we need to set up and guide the AI Harness to actually be helpful and do it right.

Let's share together how to do it!

Hybrid AI - Running AI fully locally or as private AI in the cloud

Note: If link in MicroLAB below is not functional, it's currently refurbished with an updated tech stack to a new version.

Running AI inference for hundreds of users and thousands of agents at real enterprise scale is not easy at all - you will meet a lot of trilemmas, so I am mapping my experiences in these projects:

Private AI is these days also many times one of HARD requirements in meaningful benchmarking, evals or red teaming, below are my technology demonstrators realised with AI coding assistants (I assist them) on Local AI or Private AI:

refeed.raptor.today/ GH - Looped hybrid RAG: The cost of a second look: local models, visible traces, published metrics. truth.parallax.observer/ GH - GenAI Persuasion: Four observers, four bodies of world knowledge, one RAG engine that argues the opposite of what you believe. doolally.trollino.rodeo GH - Don't feed the trolls: a durable high-risk text classifier with local LLM + SystemOne with live GUI. mem.codedorf.dev GH - Deep dive into memory for AI agents: what to store, where to store it, and whether it improves the second run.


  • inside.cottonbox.cc GH - Sandboxing is wild: Sheep pen, Cattle pen, Paddock for horses or even Metal cage for wild animal?
  • mcpwith.codemode.cc GH - MCP is still reinventing itself: Now we are executing code inside MCP servers?
  • rest.graphql.win GH - REST and GraphQL + Code Mode: in observability together they are both clear winners (So use them BOTH!)
  • rainbow.redoras.dev GH - I need REDORAS: Rapid Exploitation & Deception Operations for Resilient Architecture & Systems.

  • yesmom.selectia.dev GH - My family of System One-style models fine-tuned from LFM 2.5, designed for one-pass typed decisions with calibrated probabilities.

We all will be EDUCATORs very soon

Thanks to fortune, I am ALSO a certified, after-class educator - I teach a lot of teens the basics of STEAM using Experiential Education, EEE.

Now I'm using that hard-won experience to nurture a group of AI HARNESSes - teaching them not just to execute tasks, but to think critically, see systems holistically, and deliver real engineering value outside of just coding.

The main point of Experiential Education is Learning by Doing - this is the core of the Self-Improving approach in AI Harnesses too, but they are doing it for now in a very naive way. They are not "natural learners" - we will need to help them simulate aspects like this:

  • Curiosity-driven rewards: reward the harness for exploring and asking questions, not just for finishing the task
  • Metacognitive loops: let it reflect on how it approached a problem and reuse what worked next time
  • Dynamic taxonomy: keep its mental model of the world updated as it learns new things

Specialized customized Agentic AI Harnesses

With the revelation of the minimal coding agent Pi and others, we have the possibility to assemble our own LEGO-like stack to ideally fix AI applications in our real JOBs.

Here are my setups (inspired by lazyPI) and I create a specialized architecture gritty.guidera.party GH


My work on self assembling per Domain AI Harnesses was originally inspired by NVIDIA AVO blog and initial Stanford research.

Today we see the rise of specialized DSH (Domain Specific Harnesses), where we see AI applications shifting from focused opencode to domain tailored openharness.

Rethinking AI Agents: The Rise of Harness Engineering

Watch on YouTube

Today we see a lot of NEW works about AI Harnesses, so I instruct Hermine, my hermes agent to proactively research this topic for me:






License

This project is licensed under the MIT License - see the LICENSE file for details.

Pinned Loading

  1. localai-isnot-cheap localai-isnot-cheap Public

    EDU AI LAB: Local AI is not CHEAP at ALL! - Every AI token carries a COST! How to measure them?

    Python

  2. SGLang-0.5.3-VLCache SGLang-0.5.3-VLCache Public

    Dokerized SGLang-0.5.3-VLCache build for Local AI is not CHEAP (localai.isnot.cheap) EDU AI LAB

  3. why-hybridai-rsvp why-hybridai-rsvp Public

    Private AI on NeoClouds - Large Open Weight models need a lot of VERY large GPUs (aka massive infra)

    HTML

  4. open-responses-memproxy open-responses-memproxy Public

    Open Responses demo with in-memory proxy for EDUAI LAB 'Local AI is not CHEAP' (localai.isnot.cheap)

    Kotlin