AI Lab: The Feedback Loop Behind Trustworthy AI

AI is more accessible than ever.

In the real world and behind the scenes,
it's a very different story.

That's right.

I mean, think about the built environments
as an example, right?

Conditions are always changing,
buildings are different.

And you know what we
tend to forget sometimes?

There are real people
living and working inside
of these buildings.

How do we know they're
reliable enough to deploy?

Well, that's the question that we'll have answered today because our guests actually have met that challenge.

They do not just research
AI and dream up things,
but they also build, test,
deploy, and then they do it all again.

Oh, this sounds exciting.

So where are we gonna start?

We'll start at the research with Foutse Khomh.

He's an AI researcher and
leader at IVADO and Mila.

[Dominique] I'm Dominique Silva.

[Scott] And I'm Scott Tew, and this is "Healthy Spaces," conversations at the frontier of sustainable technology.

AI is a technology with a high disruptive potential.

It will significantly transform
the way we do many things.

As this technology will permeate
the society and our activity,
trust and trustworthiness in
the technology, it's critical.

From an engineering point of view,
it means the engineer building the systems
have gained enough confidence to be able
to provide guarantee about the
functioning of the systems.

So we may not understand
all the inner working of a technology,
but we have engineering tools
that allow us to study the technology,
to test it, to interact with it,
and grow our confidence in
the behavior of the technology
to a point that we can provide
some level of guarantee
that this is likely to be
providing this kind of output.

Although the output may be
different from one query
to the next, but the output
should remain aligned
with the expectation,
and that allow us to be
able to provide guarantees
that this will provide a service
with this level of reliability.

So it's sort of a guarantee that the tool will meet expectations.

Is that what you're saying?

Exactly. Exactly.

So what if you identify a weakness or what if a user identifies a weakness?

What's the role there
for someone in your field
that works on trustworthiness?

Isn't weakness a concern?

Understanding the weaknesses and all the failing mode, it's critical to be able to ensure that the technology that we bring in productions have been calibrated, controlled in a way that the output can be reliable.

So in the field of trustworthy AI,
most of the things that we try to do,
it's, for example, we try to invent
novel techniques to ensure
that when we build AI system,
they are robust, right?

By robust mean under
unexpected conditions,
the system should still
maintain a behavior
that is aligned with our expectation.

How can we ensure that in a system,
an AI system that we build?

So this require a deeper understanding,
not only how the system
behave in distribution,
because the current form of AI,
which is heavily data driven,
usually will behave pretty
well in distribution.

The system, the training process
allow us to extract as much
regularity from the data
to which the system is exposed
and build a system that will
be fairly good in distribution.

But usually when the system is
deployed out of distribution,
we expect the systems to be facing
with inputs that to some extent
it's a bit novel to the system.

Of course, we train system to generalize,
but generalization is still a struggle
that we have in the AI field.

There's this work that's in a lab.

You actually are a partner
for the BrainBox AI Lab,
for instance, and where things get tested,
but at some point you
release them into the world
and you're expecting this tool
to perform to a certain
level of standards.

Does it always work perfectly? Not always. Of course, it's a challenge.

And part of the work that we
do is to try to bridge the gap
between these two reality.

So when we do in-lab
testing, it's very important
to provide the testing frameworks
that mimic as much as possible
the reality of the real world
in which the solution will be deployed.

In the case of vendors, for example,
if we're building a
technology for buildings,
because of the domain knowledge
of the context of buildings,
we are able to design testing framework
that's emulate as much as possible
the reality of the real-world environment.

At leveraging this framework,
we are able to exercise the
behavior of the solution
that we're constructing
as much as possible
to understand the
weaknesses, the corner cases,
and to improve of these weaknesses,
to understand the operation
domain of the solution.

So by through this process,
we can understand the envelope
in which the solution has a
certain level of robustness,
so a certain level of reliability,
that is expected when the
solution is deployed in the field.

And on the spaces where we don't
have a level of reliability
that's up to the standard
of our expectation,
there are different
mechanism that we pursue
to try to either build more
adapted AI for those spaces
or the capability of the
AI system in those spaces.

So it's really an engineering process
where deeper understanding
of the inner working of our solution,
the failing mode of our solution,
allow us to build control around
and to build a systems of systems
that allow us to back this
reliability and trust principle.

And at the same time,
you can have a systems
on which user over-trust,
which is not trustworthy.

Would you say that over-trusting is a bigger problem even?

Over-trusting is a bigger problem because over-trusting will lead to deploying solutions in context in which they shouldn't have been deployed in the right place, which means when the inevitable adversary consequences occur, this usually actually destroy the trust.

So it's important to avoid
building solution on
which people over-trust
because trust is very hard to gain,
but very easy (chuckles) to last.

Once the system divarts from
the expectation of users,
the mistrust becomes
actually more stronger,
strongly ingrained in the end users.

It's achieved through good understanding
of the behavioral system
and ability to communicate on the strength
and the limitations of the solutions
to the different stakeholders
and also the process in
which that is in place
to ensure that the behavior
is being monitored,
it's being controlled,
this process to be trustworthy
in the eyes of these
different stakeholders.

It's all these ingredients
that enable the deployment
of a trustworthy solution.

Do all these systems, all these tools, do they all have flaws?

Are they like humans
where we all have flaws,
we all have some kind of weakness,
should we view AI
and AI tools having similar weaknesses?

So I think it's depend on the type of AI.

It's evolving as we are building
capability in those systems.

That's to say, depending on the complexity
of the AI technology, we
have quality assurance
and safety tools that have
different level of maturity.

And it's also very
important for the audience
to be aware of this
because all this contribute
to the level of trust
that we should be putting
in those technology.

We had a conversation prior to today's discussion and you mentioned that think about AI as Swiss cheese.

It's great.

There are always some holes
and it's trying to figure
out what those holes are.

I thought that was really helpful analogy.

That's a big analogy behind my work, like, to me, my approach to trustworthy is to try to understand as much as possible, where are the holes in those systems?

Because even if the structure,
it's fragile in the sense
that it has all these holes,
if we know where the holes are,
we can stack this cheese in a way
that allow us to compensate
from these weaknesses
and build a strong structure.

So this allow us to build an
architecture that is robust,
an architecture on which
we can actually provide
some level of guarantees of behavior,
an architecture on which we can have
some level of observability
because observability is very critical
as when we're building complex systems.

We need to understand how
the different pieces interact
to contribute to the
behavior of the systems.

And this understanding,
it's what allow us to have
the right control in place
and the right way in place.

And this is what allow
us to build solution
that are trustworthy on which users
could reasonably put
their trust on the system.

So Foutse, what is the role of this testing in labs, maybe, like, the BrainBox AI Lab where the testing occurs?

I mean, what's the role
of collaboration there?

Somebody like you who's a
scientific director, a researcher,
what's the role of collaboration
and bringing these tools to
the public that we can trust?

In the case, for example, of BrainBox, the lab, it's now one of our shining piece in Montreal.

It's a very happy result
of a collaboration
that happened within the ecosystem
over a significant period of time
where our academic prototypes,
our academic technologies could interact
with the reality of the use case
of the industry in which BrainBox,
it's actually deploying technology.

And this information coming
from the industry field,
it's what allow us to calibrate
testing evaluation framework
to ensure that the tools
and the new algorithm
that we are designing
cope with the reality of that ecosystems.

It's allowed to get use
cases, real-world use cases,
for calibrating on
testing this technology.

So to ensure trustworthy,
we need all these players
to have these constant
interactions, right,
because the challenge
of the real environment
inform some of the
design academic challenge
that we have on a scientific perspective.

And it's allow us to expand the technology
that we're building, which,
with the connections,
it's being tested in real context.

And we have this feedback that
is coming back to the labs.

And all this, it's feeding this ecosystem
and allow us to innovate constantly
and to continue to pursue in this way.

If you look at the landscape
of AI industry nowadays,
the frontier labs that are contributing,
the most advanced
recently are industry lab.

Why?

Because they have these use cases,
because they have this
interaction with the user base.

They have this third-party
feedback that are coming in
to allow the scientific team to push,
continue to push the boundary
of the research innovation.

So those feedback loops just continue.

That's continue and that is critical.

So in our context, I think
the lab that have been opened
by Brainbox AI in Montreal,
it's a great addition to our ecosystem
and it's what will allow us
to continue innovating on this field,
allow us to continue build AI systems
that can pass the test of the real world,
the real environment.

And also for our students,
and as we train students,
they have the opportunity to
interact between these two.

We're very happy and proud
to have this addition in
our ecosystem, definitely.

And also for our students,
and as we train students,
they have the opportunity
to interact between these two
faces, the lab on the campus
and then the lab in grain,
in industrial use cases.

This is only beneficial
to allow us to continue
to contribute to this technology
that we are inventing as
we are using in the field.

Okay.

I really loved that Swiss cheese metaphor.

Mm. Me too.

I mean, he's right.

Every system has holes, right?

The skill is in finding them,
and you can only really find
them if you actually test it
in the real world and look
at where it's failing.

Yeah, I mean, but you don't see that in the lab only.

So research is important obviously,
but the real world also teaches
a lot of things about these applications.

The lab and the field
or the lab and the real world
are two different platforms,
two different stages.

And good news is there's a feedback loop.

One learns from the other
and improvements happen
over and over again.

Is that where we're heading, Scott, out into the real world?

Absolutely.

And who better to take us
there than Riaz Raihan,
who is the chief digital
officer at Trane Technologies?

So we've been planning this for over 15 months and think of this lab as a working laboratory, right?

It's not just men and
women in white coats.

It's a lot of very capable engineers
and product managers and designers
and people that really understand
how AI is used in the HVAC industry.

About two years ago,
we acquired a wonderful
company called BrainBox
because they were by far
the leader in this space.

Some of the results
they were able to drive,
tangible results, were very impressive.

Customers were confirming
that they were saving
15, 20, in some cases
even 30 and 40%, and consistently.

And then as we started integrating them
into the fabric of Trane
Technologies, we said,
"Why don't we truly make
this a showcase lab?

Why don't we leverage the
location of BrainBox?"

And BrainBox is based in Montreal,
which is one of the world's top three hubs
for AI development,
the other two being
Silicon Valley and London
and Cambridge in England.

The lab itself is a working lab.

We have real people doing real work.

And when our customers
or our prospects or
custodians of sustainability
visit this lab,
what they find is an opportunity
to truly experience
how this wonderful technology
works, to see it in action.

They can walk through the lab itself
and go meet the men and women
who are building this technology
and ask questions, interact with them,
understand what's driving them,
understand what are
the latest technologies
that they're leveraging,
understand is it predictive
AI, classical AI,
generative AI, a combination of those.

It's a lab that has a meaningful impact
and it's a lab that allows us
to truly engage with our stakeholders.

I'd love to also meet the people behind some of these AI tools and I think that's a unique moment.

So Riaz, is this, like,
an academic exercise
that we're conducting in
Montreal or is it something else?

Are we actually producing
something for the world?

This is not just academic. It's very commercial.

In Montreal, we do a lot
of fundamental research,
but then we create products,
and these products are built
for real customers in the real world.

And we ship these products to them,
we deploy these products,
we monetize these products,
and then we operate these products
24/7 for these customers.

So it's a living lab.

It's a very commercially focused lab,
and it's solving real problems
for real customers in the real world.

Yeah, Riaz, I hated that I had to miss the recent ribbon cutting opening of the lab.

I hope to get there at some
time in the near future.

Well, Scott, you're in luck because we put together a little video montage.

I know you missed it and we missed you,
but I'd love to show you
what the lab is all about.

But I thought I would
show you a few things
that we're going to talk about
tomorrow at the big ceremony.

First of all, I want you
to take a look at this beautiful lab
which has all of the cool tech
that BrainBox has brought to market.

The first station talks
a little bit about ARIA.

This is the virtual engineer
where you can actually converse with ARIA
either in text-to-text or voice-to-text
or use the pre-approved prompts
and really get responses
from ARIA in real time.

That was really cool.

The second station
is really about Trane Autonomous Controls
or Trane AI Control.

This is when you get to see a
lot of different algorithms.

And what I loved about
the approach the team used
was to have each cylinder
on that table represent
a different algorithm.

And then you could pick the
one you want to talk about,
for example, Hydra or Hercules,
and learn a little bit more about it,
but also learn how these
algorithms work together in unison
to solve problems for
complex HVAC systems.

Now, the other station
was all about Cloud BMS.

And Cloud BMS is this cloud-based
building management system
that truly brings together
the power of ARIA,
the power of AI control,
and the power of cloud-based
BMS into one package.

We saw some great demos of how
customers are using Cloud BMS
to drive tremendous value.

The next station was all
about M&V, measure and verify.

As a matter of fact, there's
some people there right now,
and this was all about understanding
how customers are using BrainBox AI,
the kind of benefits they get.

And many of these customers
have actually allowed us
to use their names and
use specific numbers
about the types of savings
they've gotten on the cost side,
but also how much carbon
emissions they have reduced.

And I know you really care about that.

And finally, there's a station
where we have a robotic arm
of an actual robot from
one of Trane's factories.

And this robot was adopted by BrainBox
and brought into the lab here
and given a place of pride.

And that's the station
where you learn a lot
about all the different digital offerings
that we have at Trane Technologies.

Last season, we had Jean-Simon on the podcast where we talked about BrainBox's flagship autonomous building controls, Riaz, which I think you know well, and I've heard you speak about this other places about the potential and the opportunity for some of these tools to truly make a difference in more sustainable operated buildings.

So what's new in that space now?

A couple of years ago, a lot of our focus was on the air side.

Customers really wanted
to leverage this on
the water side as well.

I'm happy to report
that we have algorithms
that are now making a big impact.

You know, we are looking
at differential pressure optimization.

We are looking
at the condenser water
temperature optimization.

We're looking at optimizing
the operation of the valves.

And the combination
of our traditional
chiller plant optimization
through controls and BrainBox AI
is proving to be extremely powerful.

Let me give you an example.

We have a major customer in Singapore.

Singapore is a great place
to test this kind of technology.

They ran this pilot with
BrainBox for over three months.

I think it's coming up to six months now.

And during that period, they were able
to show significant improvement
in not only the runtime of the chillers,
but also in the energy
consumed as a result of that.

And they're in the process of figuring out
the exact savings, but I can tell you
it is well above what we
expected and very impressive,
you know, we are already
in the 10, 20% range.

It could be higher.

And when customers asked for this,
I think it was great for the BrainBox team
to go off and work on it.

We now have working algorithms.

We are in the process
of a commercial rollout.

We are starting with some of
the most important markets
for us in the United States and Europe.

So you're gonna hear a lot about this.

There's a lot of promise that AI has for the world, and maybe the lab that you mentioned at the beginning of this discussion is testing things and we see great things in a lab setting, but what happens in the real world?

Can we trust the things that,
I mean, you just gave some
figures about Singapore,
a 10 to 20% improvement.

Can customers, can building managers
trust these tools that are AI related?

I think if done properly, yes, they can, and they should.

Our approach is founded on
three key principles, okay?

The first principle is secure by design.

This is very important.

When we start designing algorithms,
when we start designing solutions,
security is the first thing
we look at, not the last.

We start with security, data security,
physical access security,
identity security, et cetera.

So we start with that.

The second is strong compliance
and going beyond compliance
with every regulation that
exists across the globe.

The third is partnerships.

We are part with some of the
world's leading organizations.

We are working with some
world-class organizations,
and these folks are creating
amazing capability around
AI trust and AI security.

So if you put those three things
together, secure by design,
which is a choice, I call
that the triangle of trust,
and it allows us to truly put
our customers in the middle
and put the triangle around them,
make sure we are building
trust from day one.

There's a open secret, I think, with AI.

I think a lot of people have heard
that hard to make any
money with it though,
not even sure if companies
are making money with AI.

So what's your take on that?

If you look across the AI ecosystem globally of the hundreds of thousands of AI companies that have mushroomed over the past four to five years, less than 5% have managed to monetize their technology.

When we acquired BrainBox,
they were one of the few companies
that were actually monetizing
their software, their AI.

So they are in that 5%.

What makes this company successful
and what puts it in that 5%
when everybody else is
not able to even monetize?

It turns out, Scott,
I think there are two important factors.

Factor number one,
you need to be able to
solve a specific problem
that actually has a monetary payback.

And if you can do that, you
have a much better chance
of customers wanting to
pay for that solution.

Number two, your solution itself
has to be very focused
on a specific domain.

Just leveraging general purpose
AI that can do many things
doesn't tend to lead to
successful AI companies.

Having an AI company that
does something very well
and, you know, very specific
and solves it with domain expertise,
that seems to be the holy
grail of successful companies.

And we see that as a trend.

So BrainBox obviously is
optimizing HVAC operations,
and that's a very specific domain.

They bring a lot of domain expertise.

They're doing it really well.

So they picked a problem worth solving
and they're solving it really well, right?

So those two things come
together, they're monetizing.

The other piece is companies
that solve problems at scale
tend to be far more successful
than companies that are trying
to solve very small problems.

So to come back to your original question,
I think those are the
reasons why monetizing
is very difficult and then
break even is the next bar,
and then after that is
actual profitability.

So those are kind of the
stops along the journey,
but getting to monetization
is not trivial.

You're sort of speaking my language as a sustainability leader, helping solve something, helping something be more productive, helping something operate more efficiently.

Can we talk about a
sustainability-related concern
that a lot of our
listeners and viewers have?

And that is around the
amount of energy and water
that it takes to run these data centers.

What are your views on that,
and where do you think this is going?

The data center discussion is real.

I mean, data centers
are a massive consumer
of electric power, no question,
and of water.

That's right.

And of land.

I've been inside and on top
of and around data centers
for a long time.

And you learn a lot when you spend
that much time in data
centers and working with them.

So let's start with the positive.

The first thing one has to
remember is data centers
have solved a real problem for humanity.

They have created an
infrastructure that has allowed us
to bring computing to the masses
at a cost which is acceptable.

As we get into the AI age, the good news
is that data centers
are creating capability
that can solve the
problem that they create.

So if you look at the total amount
of energy the world uses globally,
the EPA publishes these numbers
and you can actually go and see
and you kind of figure out
how much of that energy is used for HVAC.

It's a large chunk, about 10% globally.

Now, you have a number x.

Question, with technologies like BrainBox,
if we were to apply them globally,
which is an AI technology,
how much of that x could you save?

Well, you could save at least 20 to 30%.

Let's take 20%.

So you're saving 20% of x.

What is that number?

Well, that's a big number.

That number is more than all the power
used by every data center
on the planet times four.

So you pause and think about that.

This is an AI solution born
and resident in a data center,
or in data centers,
that can help humanity
save four times more energy
than all data centers combined
if applied at scale consistently.

That's a big if.

But the point of this thought experiment
is to look at the
opportunity and to say, "Wow,
that is the power of AI."

So while AI, you know, is vilified
often and for all the right reasons,
I do believe that there's
also massive opportunity
in looking at this space as a solution
to solve the problem it creates.

Because if you do solve
the problem, Scott,
if you kind of step back for a minute
and you look at 5, 10 years from today,
let's say we apply this
kind of technology,
the world needs more HVAC.

And as HVAC becomes more prevalent,
especially in the Eastern Hemisphere,
the world will consume
more energy around HVAC,
and that represents a massive opportunity
for us as custodians and
stewards of this industry
to leverage AI solutions to
reduce their consumption.

And eventually we could get to a point
where the reduced power
actually equals or is better
than the consumed power.

And that is truly the
promise of such technologies.

Well, look at that.

We have just come full circle.

This has been the journey
on this episode, right?

We started with Foutse talking
us through the research
and the testing and the
Swiss cheese analogy,
and here's Riaz coming and talking to us
about real-world deployment.

I've walked away
realizing that the AI lab,
it isn't just about theory.

It's about building and
testing real solutions
that can be deployed immediately
and really help solve some big challenges.

And this is the proof of the
power of partnership, right?

Yeah, and that is the bigger picture here.

I mean, connecting sustainability goals
with operational outcomes,
this is where we're going to
see the big impacts happening,
and that's where things
get really interesting.

I mean, this is linked to
Riaz's thought experiment
where energy that's not
used is the best path.

HVAC is roughly 10% of the
world's energy consumption.

And if we apply AI tools to that issue,
then we're going to save
four times the energy
that data centers consume globally.

I mean, that is the amazing moment.

This has been "Healthy
Spaces" with me, Scott Tew,
and my co-host Dominique Silva.

We're back in two weeks
with another episode,
so be sure to like and
subscribe so you don't miss out.