Alessandro Secchi

HITACHI Sustainability Award (06.02.2025, London)

I recently received the Inspiration of the Year Global Award (IYGA - section Sustainability). A good opportunity to take a break from the usual working routine and to reflect on the meaning of sustainability in the Energy and AI transition.

🌱 Sustainability means electrifying everything we can, increasing the quota of renewable sources, and investing in solutions to make the grid more flexible.

💡 HITACHI has a broad portfolio both in IT and OT which could lead this transition, using AI to increase the employee's productivity, to find synergies and new ideas in order to generate the so-called "social innovation business".

However, the direction set by most of the high-tech companies might not be the best course of action to ensure a sustainable future. That's why I have written down some general thoughts about the trajectory of this transition:

Total factor productivity growth rate ↓

Measured TFP growth has slowed in many advanced economies compared to the mid-20th century. While the digital revolution has enhanced productivity, its effects are less directly visible in traditional TFP metrics than those of earlier, more disruptive innovations (electricity, motor vehicles, indoor plumbing...).

GDP growth rate ↓

GDP growth has slowed in many advanced economies. Of course, this moderation reflects not only slower TFP growth but also factors like demographic trends, climate and energy crisis, and economic maturity.

Unemployment growth rate cyclical

The relationship between technological progress and unemployment is complex because it involves both job destruction and job creation, as well as broader economic and social factors. The net effect on employment depends on how well societies adapt to these changes through education, policy, and innovation.

Historically, technological progress has not led to mass unemployment. For example, the Industrial Revolution initially caused job losses in agriculture but eventually created millions of new jobs in manufacturing and services.

However, the AI revolution is something new. If AI adoption is unchecked, wealth concentration may increase, job displacement could dominate, and the natural rate of unemployment may rise permanently.

Why?

Productivity gains are no longer significantly enhancing the production function of modern economies in the way they once did. In the 1940s, innovations like electricity use empowered workers to achieve higher output by revolutionizing industrial processes. By the 2000s, computers primarily redirected labor toward new sectors (e.g., IT, digital services) rather than universally boosting productivity.

Today, AI-driven automation represents a further shift: it aims to replace human roles (e.g., computer use, operator, agents...) rather than augmenting human capabilities or creating entirely new industries. Perhaps, robotics and autonomous systems (e.g., self-driving cars, warehouse robots) will create massive new industries, but unlike past technological leaps, these sectors may not generate proportional human employment. The "big new AI industries" could profit only a few tech giants, highly profitable but labor-light.

This divergence risks a "productivity-prosperity paradox": GDP and tech corporate profits may grow, but labor's share of income could decline, reducing the "shared prosperity".

What's next?

Considering the fast pace of AI development, and the pursuit of AGI from many tech leaders, I expect to see in the next 5-10 years, an increase in GDP growth, as well as an increase in unemployment.
This might force governments to act on this AI revolution with new policies against complete automation.
That's why I'm developing and exploring the use of personal AI assistants to augment human abilities and unleash creativity for open innovation from the bottom up.
A future where computers act as "bicycles for the mind". See the picture below where the "man on a bicycle" represents the most energy-efficient form (per unit of body weight) of moving on earth. Augmentation instead of automation.

Some pictures about the awards ceremony in London:

Back from Anglogold commissioning in Tanzania (03.2024)

On my way back home, I tried to write down my reflections about the past three months in Tanzania, where we supported the partial electrification of the Geita Gold Mine.

𝐒𝐨𝐦𝐞 𝐟𝐚𝐜𝐭𝐬 𝐚𝐛𝐨𝐮𝐭 𝐭𝐡𝐞 𝐩𝐫𝐨𝐣𝐞𝐜𝐭:

🏭 The mine currently receives 40MW of power from 4 Wärtsilä diesel generators located on-site.

🔌 In 2020, the mine embarked on a grid integration project to build a 33/11kV 60MVA substation connected to the national 220/33kV grid at Mpomvu village, which will also provide power to other 130 villages in the region.

💡 Given the complex operational requirements of the process plant and the weak grid, there is a crucial need for a reliable and secure power supply. That's why a Static Synchronous Compensator is necessary to provide dynamic voltage compensation to support the 33kV bus during grid faults and to regulate its power factor.

🔋 With about 235 GWh/year of energy demand and an average CO2 emissions from the electrical grid of 0.27 kg/kwh (powered by 35% of renewable energy), this project will bring a reduction from 0,14 Mt to 0,063 Mt.

💸 The mine's annual energy expenditure will drastically decrease, with a NPV of 26 million, IIR 46% (over 10 years) and payback period of the investment in less than 2 years.

𝐒𝐢𝐧𝐜𝐞 𝐭𝐡𝐢𝐬 𝐢𝐧𝐯𝐞𝐬𝐭𝐦𝐞𝐧𝐭 𝐬𝐞𝐞𝐦𝐬 𝐬𝐨 𝐚𝐩𝐩𝐞𝐚𝐥𝐢𝐧𝐠, 𝐢𝐭 𝐦𝐚𝐝𝐞 𝐦𝐞 𝐬𝐞𝐚𝐫𝐜𝐡 𝐟𝐮𝐫𝐭𝐡𝐞𝐫:

🌎 OCSE countries are accountable for 1/3 of the C02 emissions and in the last 10 years they had a yearly reduction of about 1%.

🔥 Global Emissions continue to rise, reaching a new record of 37.4 Gt in 2023.

💸 Out of $1.8 trillion of clean energy investments in 2023, developing economies account for about 15% (and they represent roughly a third of global GDP and two-thirds of the world's population).

⛏️ Critical minerals are pivotal for the energy transition and areas like Africa owns over 40% of global reserves.

𝐈𝐧 𝐜𝐨𝐧𝐜𝐥𝐮𝐬𝐢𝐨𝐧:

🟢 I wonder if substantial investments in the electrical grids of OCSE countries is the fastest and most effective way to addressing the global climate crisis and achieving the famous net zero emissions.

🟢 Perhaps exporting cutting-edge technology and expertise to developing countries, that are in the middle of their industrial revolution, could offer a possible answer to the "hard problem" of the energy transition.

🟢 Wouldn't it be better to reconsider (and readapt) the past approach of clean development mechanism CDM from the Kyoto protocol to valorize these investments?

...Back to my experience in Tanzania 🇹🇿. The humbleness of the people I have met and their respect for nature have thought me that this energy transition might have even more relevance in those places, where nature is still part of daily life and people know how to praise it. As they told me as soon as I arrived at Mchauru Village: "Welcome back to the reality". 😄

🏭 Investing in developing countries could be an effective way to fight global warming. However, at the same time, we should work to reduce greenhouse gas emissions per capita in places like the US and Europe, where levels are much higher than in the rest of the world.

𝐀𝐧𝐝 𝐭𝐡𝐢𝐬 𝐦𝐢𝐠𝐡𝐭 𝐛𝐞 𝐚𝐧 𝐞𝐯𝐞𝐧 𝐛𝐢𝐠𝐠𝐞𝐫 𝐩𝐫𝐨𝐛𝐥𝐞𝐦:

💸 Direct investments in the transmission and distribution system, as well as in the ancillary services (primary and secondary markets, capacity markets, FACTS, etc.) need a clear evaluation and communication to the citizen since it might lead to substation increase in their bills.

🌄 The case of the German corridors in striking. The costs associated with transmitting high-voltage energy from wind generation in the North to consumption areas in the South amount to several tens of euros per MWh. These costs should be added to the production price at auctions (so-called "grid parity") and also include additional surcharges to maintain the quality and safety of the electrical service.

OCSE countries should take the lead in this energy transition as they can reduce the so-called green premiums by investing in new innovations.

🟢 Perhaps promoting and reviewing the concept of grid parity, which cannot solely refer to the local cost of production but must include additional costs to the electricity system. For example, nodal pricing, which redistributes some costs associated with the location and intermittency of PV production could be considered. This would foster the rise of Virtual Power Plants (VPPs), where by aggregating the interests of around a hundred households into a mini plant of a few hundred kilowatts, the cost of producing one kWh could be reduced by two-thirds (kind of car pooling but with energy ☀️).

Let me know your thoughts 😊

This is an excerpt from an interview I received due to the Hitachi Sustainability Award. Here, I aim to highlight the project's positive impact on the environment and local communities and summarize some key steps I believe are important to tackle the Energy Transition. I also share a few wishes and aspirations for the people at the forefront of this transition, along with some ideas for my own professional growth.

And this is a funny video I made during my 3 months on site 🫠.

How to Avoid a Climate Disaster

We know that the Energy Transition is quite a complex problem, but the book "How to Avoid a Climate Disaster" helped me to grasp it in smaller and clearer solutions, which I try to summarize here:

🏭 The world typically adds 52 billion tons of greenhouse gases to the atmosphere every year.

This includes:

• 30% from "making things" (mainly cement, steel, plastic)
• 26% from "plugging in" (electricity)
• 21% from "growing things" (plants, animals)
• 16% from "getting around" (planes. Truck, cargo ships)
• 7% from "keeping warm and cool" (heating, cooling, refrigeration)

🌍 The world needs to provide more energy without releasing any greenhouse gases so the poorest can thrive.

👪 The population growth goes down as we improve health.

⚡️ Electricity demand will increase also driven by new ways of making cement and plastic.

𝐈𝐟 𝐰𝐞 𝐰𝐚𝐧𝐭 𝐭𝐨 𝐬𝐮𝐦𝐦𝐚𝐫𝐢𝐳𝐞, 𝐭𝐡𝐞 𝐩𝐚𝐭𝐡 𝐭𝐨 𝐳𝐞𝐫𝐨 𝐞𝐦𝐢𝐬𝐬𝐢𝐨𝐧𝐬 𝐜𝐨𝐮𝐥𝐝 𝐛𝐞:

🟢 Electrify every process possible and explore advanced biofuels and electrofuels (aim for zero-carbon cement, steel, fertilizer and plastic).

🟢 Get that electricity from clean energy sources (including next generation nuclear fission and nuclear fusion).

🟢 Use carbon capture to absorb remaining emissions (possibly directly from power plants, as some national agencies are already demanding).

🟢 Discuss and understand the impact of geoengineering (brightening clouds and injecting fine particles into the atmosphere).

🟢 Government policies to close the gap, as energy businesses spend an average of 0.3% on R&D compared to the 10% of IT and pharma industries.

🟢 Adopt more F-gases-free coolants

𝐀𝐥𝐥 𝐭𝐡𝐞𝐬𝐞 𝐩𝐨𝐢𝐧𝐭𝐬 𝐝𝐞𝐩𝐞𝐧𝐝 𝐨𝐧 𝐚 𝐬𝐢𝐧𝐠𝐥𝐞 𝐟𝐚𝐜𝐭: 𝐥𝐨𝐰𝐞𝐫 𝐭𝐡𝐞 𝐠𝐫𝐞𝐞𝐧 𝐩𝐫𝐞𝐦𝐢𝐮𝐦𝐬.
𝐓𝐡𝐚𝐭'𝐬 𝐰𝐡𝐲 𝐎𝐂𝐒𝐄 𝐜𝐨𝐮𝐧𝐭𝐫𝐢𝐞𝐬 𝐬𝐡𝐨𝐮𝐥𝐝 𝐭𝐚𝐤𝐞 𝐭𝐡𝐞 𝐥𝐞𝐚𝐝 𝐢𝐧 𝐭𝐡𝐢𝐬 𝐞𝐧𝐞𝐫𝐠𝐲 𝐭𝐫𝐚𝐧𝐬𝐢𝐭𝐢𝐨𝐧 𝐚𝐬 𝐭𝐡𝐞𝐲 𝐜𝐚𝐧 𝐫𝐞𝐝𝐮𝐜𝐞 𝐭𝐡𝐢𝐬 𝐝𝐢𝐟𝐟𝐞𝐫𝐞𝐧𝐜𝐞 𝐛𝐞𝐭𝐰𝐞𝐞𝐧 𝐜𝐮𝐫𝐫𝐞𝐧𝐭 𝐬𝐨𝐥𝐮𝐭𝐢𝐨𝐧𝐬 𝐚𝐧𝐝 𝐭𝐡𝐞𝐢𝐫 𝐜𝐥𝐞𝐚𝐧 𝐚𝐥𝐭𝐞𝐫𝐧𝐚𝐭𝐢𝐯𝐞𝐬 𝐛𝐲 𝐢𝐧𝐯𝐞𝐬𝐭𝐢𝐧𝐠 𝐢𝐧 𝐧𝐞𝐰 𝐢𝐧𝐧𝐨𝐯𝐚𝐭𝐢𝐨𝐧𝐬.

Let me know your thoughts 😊

AI, LLM and Energy (07.2024)

A few months ago, I wanted to start some AI projects to better understand its potential and see if it could really assist me.

1) Regarding foundation models (LLMs), nearly every week, there was an announcement about new, more powerful models from different companies. Sorting them out was quite straightforward:
📖 Open-source models (>70B parameters):
Llama (Meta) vs. Grok (xAI from Elon) vs. Mistral AI (largest European LLM) vs. Falcon (Abu Dhabi TII)

🔐 Private models (>70B parameters):
GPT (OpenAI/Microsoft) vs. Gemini (Google) vs. paLM (Google) vs. Claude (Anthropic) vs. Olympus (Amazon) vs. Ernie (Baidu, Inc. - China).

2) Then I started to look into open-source models that allow me to customize my application and maybe run it on my local machine.

💻So, I began exploring LLMs that can run locally, using tools like: LMStudio vs. Ollama vs. GPT4All.

Of course, my laptop can only handle LLMs with <7B parameters, but for initial experiments, I thought this would be more than enough.

3) The next step was to look for basic LLM frameworks and repositories from this new AI world:

⛓ From Langchain and LlamaIndex (for Python and JavaScript) to Relevance AI and Make.com (for low-code solutions).

As a good starting point, I decided to explore some pre-built open-source projects and here, I was astonished by the thousands of open-source projects publicly shared every day on GitHub and Hugging Face (the GitHub for LLMs).

4) Among them, I tried:

AutoGPT (linked with OpenAI) vs. HuggingGPT (integrated with models from Hugging Face) vs. AutoGen (a multi-agent project that supports function calling to extend the capabilities of the model by generating documents, sending emails, etc.).

Testing these projects was inspiring, but I must be honest. They are still prototypes with bugs that require new daily releases.

Naturally, their closed-source counterparts are more reliable:

Azure AI Studio (a Microsoft integrated environment to develop AI agents) vs. Microsoft 365 Copilot (quite useful with Excel and Outlook) vs. OpenAI GPTs (arguably the most utilized AI playground).

It's also worth mentioning AI Image and Video Generation models (DALL-E 3, Midjourney, Stable Diffusion, Sora, Runway). See the breakthrough experiment from Reid Hoffman.

5) Finally, I opted for an AI open-source IDE that allowed me to develop and connect objects (nodes and graphs). Tools like Flowwise, Langlow, but especially Rivet, are great for visualizing and building complex chains and create production-ready applications.

𝐒𝐨𝐦𝐞 𝐟𝐢𝐧𝐚𝐥 𝐫𝐞𝐟𝐥𝐞𝐜𝐭𝐢𝐨𝐧𝐬 𝐟𝐫𝐨𝐦 𝐭𝐡𝐢𝐬 𝐀𝐈 𝐣𝐨𝐮𝐫𝐧𝐞𝐲:

🟢 As of today, I don't see disruptive ideas from these AI models that could significantly change my daily life or a7 traditional company's business.
However, there is a remarkable boost in R&D and academic activities, and a huge benefit for developers who can use AI copilots to build their software or create simple websites (https://secchialessandro.github.io/). For a company business, it would be interesting to explore solutions such as Azure AI Studio with some open source LLMs models.

🟢 We are moving towards highly specialized small agents in a multi-agent and multimodal frameworks. The new GPT-4o just leaves space to the imagination 🤯.

🟢 It is worth to mention the so called large action model "LAM" (see Rabbit r1). Check out the new open source project from Microsoft "Visualization-of-Thought (VoT) Elicits Spatial Reasoning in Large Language Models".

🟢 If you aim to use open-source LLMs, maintain the privacy of your data, and run useful applications, you could do so from your own laptop (with limited complexity) or from an internal server if you're a business.

🌟 I am convinced that we are getting closer and closer and I will spend some time to seek for opportunities between this new AI world and my passion in Energy.
Some friends have already introduced new methods to operate in the electricity market. For instance, Matteo Pisani developed a model to regulate the voltage from a load tap changer using reinforcement learning algorithms (reducing unnecessary tap movements, thus maintenance costs of the device).

"𝘛𝘰𝘸𝘢𝘳𝘥 𝘴𝘪𝘯𝘨𝘶𝘭𝘢𝘳𝘪𝘵𝘺, 𝘵𝘩𝘳𝘰𝘶𝘨𝘩 𝘵𝘩𝘦 𝘦𝘯𝘦𝘳𝘨𝘺 𝘵𝘳𝘢𝘯𝘴𝘪𝘵𝘪𝘰𝘯."

Hitachi Lumada AI (08.2024)

More frequently, I hear talk about highly specialized small agents in a multi-agent frameworks.

CrewAI, LangGraph, MetaGPT, LlamaIndex, Autogen are the most famous ones used in interesting application such as:
📨 Automating the process of checking new emails and creating drafts based on internal know-how.
👨‍💼 Replicating the structure of a software company (each agent is trained on their job profile), where from a single line requirement as input, it outputs competitive analysis, data structures, APIs, documents, software repository (see Pythagora 🤯)..

Working at one of the Hitachi companies, gave me the inspiration to build a multiagent framework to exploit the huge amount of knowledge in information technology (IT) business as well as the operational technology (OT), a special combination that only a few companies can afford.

Therefore, by looking at the Lumada concept, I tried to build an application with LangGraph, where each agent is trained on the respective company data (scraped from the website section: products and solutions) and the supervisor (LumadaAI) can choose a specific agent (company) to perform the RAG (retrieval augmented generation) on the training data. In addition, I have added two agents called "Researcher" and "Coder" to perform a general web search and basic data analysis calculations.



In this framework, each agent is a node in the graph, and their connections are represented as an edge. The control flow is managed by edges, and they communicate by adding to the graph's state.

With LangGraph I could create a cyclical and non-linear workflows where the agents can revisit and refine their actions based on new data, leading to more accurate and efficient outcomes. Specialization, collaboration, flexibility and scalability are the main advantages of this kind of multi-agent systems.

This is a basic prototype and many improvements are possible:

• Improve the RAG code with a self-corrective logic (which incorporates self-reflection / self-grading on retrieved documents)
• Increase the number and quality of training data
• Add an agent trained on the customer's data
• Improve the quality of the user prompt

In the example below, I asked LumadaAI to support me in the planning, building, operating and maintaining of a new hospital.
The tool is retrieving information from Hitachi Energy, Hitachi Astemo, Hitachi HighTech, Hitachi Rail, Hitachi Construction Machinery and GlobalLogic.

🟢 I believe this kind of application can enhance the general human creativity and support many jobs by reducing the complexity of the problem, combining a huge amount of data, to come up with new products and solutions.

"𝘊𝘰𝘪𝘯𝘦𝘥 𝘧𝘳𝘰𝘮 𝘵𝘩𝘦 𝘸𝘰𝘳𝘥𝘴 "𝘪𝘭𝘭𝘶𝘮𝘪𝘯𝘢𝘵𝘦" 𝘢𝘯𝘥 "𝘥𝘢𝘵𝘢", 𝘵𝘩𝘦 𝘯𝘢𝘮𝘦 𝘓𝘶𝘮𝘢𝘥𝘢 𝘦𝘮𝘣𝘰𝘥𝘪𝘦𝘴 𝘰𝘶𝘳 𝘨𝘰𝘢𝘭 𝘰𝘧 𝘴𝘩𝘪𝘯𝘪𝘯𝘨 𝘢 𝘭𝘪𝘨𝘩𝘵 𝘰𝘯 𝘰𝘶𝘳 𝘤𝘶𝘴𝘵𝘰𝘮𝘦𝘳𝘴' 𝘥𝘢𝘵𝘢 𝘢𝘯𝘥 𝘪𝘭𝘭𝘶𝘮𝘪𝘯𝘢𝘵𝘪𝘯𝘨 𝘪𝘵 𝘪𝘯 𝘴𝘶𝘤𝘩 𝘢 𝘸𝘢𝘺 𝘵𝘩𝘢𝘵 𝘸𝘦 𝘤𝘢𝘯 𝘦𝘹𝘵𝘳𝘢𝘤𝘵 𝘯𝘦𝘸 𝘪𝘯𝘴𝘪𝘨𝘩𝘵, 𝘵𝘩𝘦𝘳𝘦𝘣𝘺 𝘳𝘦𝘴𝘰𝘭𝘷𝘪𝘯𝘨 𝘰𝘶𝘳 𝘤𝘶𝘴𝘵𝘰𝘮𝘦𝘳𝘴' 𝘣𝘶𝘴𝘪𝘯𝘦𝘴𝘴 𝘪𝘴𝘴𝘶𝘦𝘴 𝘢𝘯𝘥 𝘤𝘰𝘯𝘵𝘳𝘪𝘣𝘶𝘵𝘪𝘯𝘨 𝘵𝘰 𝘵𝘩𝘦𝘪𝘳 𝘣𝘶𝘴𝘪𝘯𝘦𝘴𝘴 𝘨𝘳𝘰𝘸𝘵𝘩."

"𝘛𝘰𝘸𝘢𝘳𝘥 𝘴𝘪𝘯𝘨𝘶𝘭𝘢𝘳𝘪𝘵𝘺, 𝘵𝘩𝘳𝘰𝘶𝘨𝘩 𝘵𝘩𝘦 𝘦𝘯𝘦𝘳𝘨𝘺 𝘵𝘳𝘢𝘯𝘴𝘪𝘵𝘪𝘰𝘯."

Role-based reasoning to Find the Right Company and Generate Clause-by-Clause Reports for Tenders 🗂️

Gemini 1.5 introduced a major breakthrough in AI with its notably large context window. It can process up to 2 million tokens at once vs. the typical 32,000 - 128,000 tokens. This is equivalent to being able to remember roughly 100,000 lines of code, 10 years of text messages, or 16 average English novels 🤯.

With large context windows, methods like vector databases and RAG (that were built to overcome short context windows) become less important, and more direct methods such as in-context retrieval become viable instead. Likewise, methods like many-shot prompting, where models are provided with hundreds or thousands of examples of a task as either a replacement or a supplement for fine-tuning, also become possible.

That's why I've decided to partecipate with a friend to the "Google - Gemini long context" competition by building a public Kaggle Notebook and YouTube Videos that demonstrate a creative and usefull use case.

We decided to use the long context window to find the best company that fit your techcnical and commercial tender requirements 📚.

• A tender is a formal process, where organizations or companies invite suppliers, contractors, or service providers to submit proposals or bids to deliver a specified project, product, or service.
• A clause-by-clause is a detailed examination of this tender, that highlights the compliant and not-compliant requirements with respect to the company offer.

As this process is carried out in most of the companies and it is quite time-consuming, we believe that the Gemini's long context window introduces a novelty in this business process:

• It analyzes long technical and commercial tenders for a project.
• It assesses the compatibility of companies' products and services with tender documents.
• It finds the best company and product-service combination to execute your project, generating a clause-by-clause report with compliant and non-compliant specifications.

Notebook Structure 📓

The notebook is available in Kaggle: The Perfect Match for Your Tech and Business Needs and it is divided into different sections, each with a specific objective:

• Dataset load
• Tenders for a project are parsed, converting their information into text.
• Information scraped from various companies' websites is loaded as text.
• All text is processed by Gemini using different prompts, combining role-based reasoning, chain of thoughts, Gemini API context caching feature, and in-chat memory.

The long context window acts as a shared workspace, recording and making all user prompt outputs accessible for seamless and holistic reasoning. In today's interconnected world, where partnerships and synergies are essential to addressing complex challenges, I think we need to create a tool that enables continuous reasoning, uncovers new patterns and solutions, and minimizes the fragmentation of insights.💡

MACHINE LEARNING / AI NOTES (ETH 10.2024)

Feed Forward Networks ⚡

One key concept is the so-called universal approximation property, which already holds for shallow neural networks with arbitrary width, consisting of an input layer, one hidden layer, and an output layer..

Indeed, at the end of the eighties, it was proved by George Cybenko that neural networks with only one hidden layer and bounded sigmoidal activation functions (i.e., lim_{x → ∞}σ(x) → 1 and lim_{x → −∞}σ(x) → can approximate generic classes of functions.

Kurt Hornik showed in 1991 that it is not the specific choice of the activation function that leads to the universal approximation property, but that it works for generic continuous non-polynomial functions.

CNN vs. Standard Feed Forward Networks 🖥️

A CNN (Convolutional Neural Network) differs from a standard feed forward neural network by the structure of hidden layers.

• The hidden layers of a CNN typically consist of a series of convolutional layers.
• The word convolution is rather a convention, coming from the replacement of the matrix vector product by a sort of convolution. Mathematically, it is actually a sliding dot product.

Recurrent Neural Networks (RNNs) 🔄

RNNs are a class of artificial neural networks for sequential data processing. They exhibit a temporal dynamic behavior where new inputs can enter, making them well-adapted for modeling and processing:

• Text
• Speech
• Time series

Hopfield Networks are a type of recurrent neural network.

Reinforcement Learning (RL) 🕹️

Reinforcement Learning maximizes the notion of cumulative reward. It is one of three basic machine learning paradigms, alongside:

• Supervised Learning: Develop models to map input and output data (i.e., regression or classification).

For a given training data set (x_i, y_i, i = 1, ..., M), supervised learning means to find the parameters θ of a neural network such that a given loss function L is minimized.
Optimization Challenge
But how to deal with a non-linear, non-convex optimization problem with millions of parameters?
With stochastic gradient descent where the gradient is replaced by an unbiased estimator.



• Unsupervised Learning: Process and interpret data based only on the input (i.e., clustering).

Training with Self-Play

The parameters of the network are randomly initialized and trained by self-play reinforcement learning. You select moves by Monte Carlo tree search, which gives – based on simulated games – probabilities π for the possible moves and game outcome z.

The network parameters are trained to:

• Maximize the similarity of p (move probabilities) and π
• Minimize the error between the predicted outcome v and the game outcome z, i.e., the loss function L.

Main Definitions

• Reward r: The agent's possibility to maximize the reward over time.
• State X: Represents the environment. U is the action space.
• History: Considered in state x → Markov state.
• If the agent can directly measure the full environment state, then it is called a Markov Decision Process (MDP).
• Policy π: The agent's internal strategy for picking actions.
• State value function (v-function): The expected return of being in state x following policy π.
• Action value function (q-function): The expected return of being in state x_k, taking action u_k, and following policy π.

So, the objective is to estimate the v function and q function.

Model-Based vs. Model-Free RL

• Model-Based RL: When the model for the environment and reward function is known (e.g., a Markovian model to predict what happens inside the environment).
• Model-Free RL: When model dynamics and rewards are unknown to the agent. The goal is to find the optimal policy while learning the model and rewards (with exploration and exploitation).

Typical Method

Typical methods involve Discrete Time Finite Markov Decision Processes (MDP).

P is the state transition probability matrix.

• In simple terms: v^*(x) tells the agent how much reward it can expect if it starts in state x and always makes the best choices afterward.
• In simple terms: q^*(x, u) tells the agent how much reward it can expect if it starts in state x, takes action u, and then makes the best decisions afterward.

Considering that:

• An optimal policy must deliver the maximum expected return being in a given state.

So the equations can be rewritten as follows:

• If the environment is exactly known, solving for v^* or q^* directly delivers the optimal policy. Once the values are known, the optimal policy can be derived by always choosing the action that maximizes these values.
• Otherwise, in model-free scenarios, use value iteration algorithms (e.g., Q-learning or neural fitted Q-learning or deep policy-based algorithms).

Q-Learning

The agent explores the environment, tries different actions, and updates its q(x, u) values using this formula:

In Neural Fitted Q-Learning: Instead of maintaining a table of q-values for each state-action pair (which can be huge in complex environments), a neural network is trained to predict these values.

1. Collect data first in an "experience replay buffer."
2. Then train from the target neural network, get the loss function, and backpropagate to train the actual Q network.

Deep Policy-Based Algorithms

• Policy-based algorithms directly learn the policy (the best actions) instead of learning the v^* or q^* functions first.
• The agent uses a neural network to directly map states to actions, optimizing the policy to maximize rewards. 🎉

Generative AI

Generative AI is typically powered by deep neural networks, especially transformer architectures, which excel at processing large datasets and understanding complex patterns.

Step-by-Step Through the Transformer with "Cat" 🐱

1. Input Embedding and Positional Encoding

• Each word in the sentence "The cat sits" is first converted into an embedding vector. For example, the word "cat" could be transformed into a vector like: [1.2, -0.8, 0.6, …]
• Then, positional encoding is added to this vector to capture the position of "cat" in the sentence. After adding positional encoding, "cat"'s vector might look like: Embedding + PosEnc = [1.4, -0.7, 0.5, …]

2. Self-Attention Mechanism 🤔

• Generate the Query (Q), Key (K), and Value (V) vectors for each word. For "cat," let's say:
  • Q_cat = [0.2, -0.3, 0.5, …]
  • K_cat = [0.1, -0.4, 0.7, …]
  • V_cat = [0.3, 0.2, -0.6, …]
• Calculate the attention score for "cat" in relation to other words in the sentence using the dot product of Q_cat with each word's Key vector:
  • Score_{cat, the} = (Q_cat ⋅ K_the) / √d_k
  • Score_{cat, cat} = (Q_cat ⋅ K_cat) / √d_k
  • Score_{cat, sits} = (Q_cat ⋅ K_sits) / √d_k
• Apply Softmax to the scores to get attention weights, indicating how much "cat" should "focus" on each word. Suppose the results are: Weights_cat = [0.1, 0.8, 0.1]
• The attention output for "cat" is a weighted sum of Value vectors: Attention Output_cat = (0.1 ⋅ V_the) + (0.8 ⋅ V_cat) + (0.1 ⋅ V_sits)

3. Multi-Head Attention 🎯

• With multiple heads, "cat" has different sets of Q, K, and V vectors per head, capturing various semantic aspects. Outputs from each head are concatenated and transformed to merge back together.

4. Feed-Forward Network 🔄

• The attention output for "cat" passes through a feed-forward network to add complexity: FFN_cat = ReLU(W₁ ⋅ Attention Output_cat + b₁) W₂ + b₂

5. Adding and Normalizing 🌐

• A residual connection and layer normalization are applied: Output_cat = LayerNorm(Embedding_cat + FFN_cat)

Putting It All Together 🧩

After multiple layers, "cat" will be represented as a vector that captures its meaning in the sentence "The cat sits." This final vector is used for the next steps, like classification or generation. Each layer helps the model build richer contextual understanding, making "cat" aware of its relationship to "sits" and "the" based on the surrounding words and overall sentence meaning.

Training Objective 🎯

For language models, the common training objective is to minimize the cross-entropy loss between the predicted sequence and the actual sequence.

ML in Robotics

Machine Learning (ML) plays a crucial role in robotics, bringing more flexibility and adaptability to robotic systems. Let's break down some core components of ML in robotics, including key concepts, challenges, and potential future directions.

📊 Formulate

• Predicting or identifying patterns in data to understand and improve robotic functions.

🔍 Formalize

• Representation: Choose models that represent data patterns well, such as:
  • Linear regression
  • Random forests
  • Neural networks
• Evaluation: Measure model performance to ensure its effectiveness in tasks.
• Optimization: Maximize the model's performance on the data given for the best outcomes.

🚀 Deploy

• MLOps: Establish robust machine learning operations to ensure efficient deployment, monitoring, and scaling of models in robotics.

🤖 Control Strategies in Robotics

Classical Control
Breaks down the problem into planning and control phases, simplifying robotic task management.

Reinforcement Learning (RL)
Directly optimizes task-level objectives and leverages domain randomization to cope with model uncertainty. This allows the system to discover more robust control responses for better task handling.

End-to-End Learning Control
Integrates all steps seamlessly:

Sensors ➔ Perception (learning) ➔ Planning (learning from data) ➔ Control (learning)

🧠 Advanced Learning Techniques

• Optimal Control: Focuses on finding the best control policy for the robot.
• Imitation Learning: Observe an expert and imitate their actions, such as observing the status of a mobile "Aloha" system.
• Privileged Imitation Learning: Uses expert knowledge for more efficient learning.

Policies
Explicit, Implicit, and Diffusion Policies: Different methods of defining the robot's behavior policy.

Kalman Filter
A mathematical method to estimate the internal state of a process, aiding robots in tracking and estimation tasks.

RL Approaches

• Model-free vs. Model-based RL: Model-free relies on trial and error, while model-based uses a model of the environment.
• Inverse RL: Learn the reward function by observing an expert. Once learned, it uses RL to create a policy based on that reward function.

🤔 Challenges in Building Generalist Robots

• Unstructured and unpredictable environments
• Generalization across embodiments – requires versatile hardware
• Contextual understanding and high-level reasoning
• Multi-tasking and priority balancing – goals and targets can be hard to specify
• Regulatory, safety, and cost-related concerns
• Limited robotic data, unlike the abundant web-scale text and image data
• Constraints of current hardware and simulators
• Requirement of dynamic models – many scenarios are hard to model accurately

Enhancing Robot Perception with AI

Vision-Language Models
For advanced robot perception, merging vision and language.

Language-Conditioned Imitation Learning
Enables robots to imitate tasks based on language instructions.

Future Directions in Robotics Control

Hybrid Control Strategies
A hybrid control strategy leverages both a model predictive controller and a learned policy for optimal robotic control and adaptability.

🤖 Robot Morphology

Exploring which morphologies (structural designs) are best suited for specific tasks and environments.

With advancements in ML and AI, robotics continues to move towards creating versatile, intelligent systems that can operate reliably across diverse scenarios and environments.

🚀 Check Out My Project on Hugging Face

I tried to solve a classical project in ML involving the LunarLander environment using the Proximal Policy Optimization (PPO) algorithm. In this task, an AI agent is trained to control a spacecraft and land it accurately on a lunar surface, all while managing its limited fuel and avoiding obstacles.

👉 You can view my project and try it out on Hugging Face: ppo-LunarLander-Ale

Ethics of AI: Fairness, Trust, Trustworthiness, Transparency

In the development of AI, ethics play a crucial role in guiding the values and principles that underpin AI systems. Key concepts in this area include Fairness, Trust, Trustworthiness, and Transparency, all of which are essential for building AI that respects human rights and societal norms.

🌐 Ethics, Trustworthy AI, and Responsible AI

• Ethics of AI: Defines the values we aim to uphold through AI systems, ensuring they align with human values and rights.
• Trustworthy AI: Guarantees that these ethical values are technically implemented and respected throughout the system.
• Responsible AI: Governs and enforces the ethical and trustworthy design, ensuring consistent application in real-world scenarios.

📜 Main AI Regulations

• 2019 – Ethics Guidelines for Trustworthy AI: A foundational document outlining core ethical principles in AI.
• The European Union AI Act: A regulatory framework focused on ensuring AI systems meet strict ethical and technical standards.

🔍 Transparency in AI

Transparency is a key aspect of ethical AI, helping users understand and trust AI systems. It includes three major components:

• Traceability of Processes: Keeping a record of all decisions and actions taken by AI systems, which helps track and understand their behavior.
• Explainability of Machine Learning Models: Providing clear explanations of how AI models reach their conclusions, making the decision-making process more u
  
  
  

  Helpful Scripts for these notes

  👉 Check out my GitHub: LLM_Basics

  
  📁 Repository Contents:
  • Experimental_Agents_ETH.ipynb
  • Finetuning_ETH_Basics.ipynb
  • Neural_Networks_ETH.ipynb
  • Prompt_Engineer_ETH_Basics.ipynb
  • RAG_ETH_Basics.ipynb
  • Supervised_Learning_ETH.ipynb

Leadership, Motivation, Inspiration

"The most beautiful experience we can have is the mysterious. It is the fundamental emotion that stands at the cradle of true art and true science. He to whom this emotion is a stranger, who can no longer wonder and stand rapt in awe, is as good as dead: his eyes are closed."

Over the past month, I've been reminded of the importance of certain emotions: inspiration and motivation, which are fundamental drivers for uncovering life's mysteries and fostering innovation. Some events, have triggered few reflections that I'd like to write down to better organize my thoughts.

Once I've read that the last metamorphosis of the spirit is represented by the "child", as a symbol of creativity, spontaneity, playfulness and openness. At this stage, we begin to wonder about everything again but with a fresh perspective. What are the unresolved mysteries of life? Which goals can we set for ourself? How can we inspire others?..

"He who has a why to live can bear almost any how."

Thus, we start exploring different aspects of life, trying to push beyond apparent limitations. We live multiple stories, encounter new cultures, begin new studies - driven by the desire to fill a void or bridge a gap.

We shall be always careful to use knowledge as a means, rather than pursuing it for its own sake. Knowledge should be disinterested, not driven by a "will to power," as it can sometimes disconnect from the more vital, instinctual aspects of life - passion, love, and creativity.

But how can we sustain these important aspects of life?

Needs can evolve into drives, which may result in motivation. In other words, to maintain motivation, we must leave some needs unfulfilled at all times.

The goal - at least for me - is to remain in the state of self-actualization: what I can be, I must be. I don't want to stray too far, but this state acknowledges the concept of linking justice to virtue (Aristotelism). Society should not just aim to maximize welfare (utilitarianism) or respect individual rights (libertarianism); it should also promote the moral character of individuals.

I like the Maslow pyramid, as it highlights the importance of professional development and equal opportunities.

Finally, a list of some quotes that often come to mind when it's about being inspired. They may seem a bit trivial or loosely connected, but I believe they serve as valuable fuel for finding certain answers:

• I think it's possible for ordinary people to choose to be extraordinary.
• Your time is limited, so don't waste it living someone else's life.
• Everything around you that you call life was created by people no smarter than you, and you can influence it.
• I think it's very important to have a feedback loop, where you're constantly thinking about what you've done and how you could be doing it better (togheter).
• The greater the risk, the greater the reward.
• People don't buy what you do; they buy why you do it.
• It's not important which company you work for, but which leader.
• Most people don't get experiences because they never ask.
• If you haven't found the right friend or partner you are looking for, it is just because you haven't talked enough with different people.
• What you do with your day, is how you live your life.

I want to close this train of thoughts with a reflection from one of the last books I read: Outliers: The Story of Success.

Success is a story of people who were given a special opportunity to work really hard and seized it, and who happen to come of age at a time when that extraordinary effort was rewarded by the rest of society. It is a product of history and community, of opportunity and legacy of the world in which we grew up. We need to be aware of all these variables so as to replace the arbitrary advantages with a society that provides opportunities for all.

Here are some of the sources of inspiration I mentioned earlier:

• The kickoff of my studies in management, economics and energy policy.
• Training sessions for Hitachi Energy with the service engineers from around the world in Switzerland.
• A volleyball weekend training event with team-building activities.