Core ideas

I want to dedicate my life to investigate artificial life, and the emergence of intelligence. This page will serve as a laydown for the core ideas, around which I want to organize my work and next steps.

Most of these ideas emerged during my master's and PhD experimentation.

Memory, Language and Intelligence

One of my main interests is to understand the role in memory in intelligence. Memory, when discussed in the context of machine learning, normally refers to hidden state of a neural network (if you are old fashion), or perhaps to the attention mechanisms in transformers (if you are new fashion).

But, I don't think that is the full story.

Memory can take different shapes. Languages is a form of memory: each word can condense a lot of information that were built over the generations. Artifacts designed by humans store information, in terms of their shape (a shape that communicates function), and/or the function itself.

Language is a compressed memory of the collective wisdom. It is the key ingredient in a neural network (the fact that the neurons are connected together, and communicating via values governed by activation functions and learned weights), but it is not that well-understood in in intra-agent communications. It is the key ingredient in a society (the fact that people are connected together, and communicating via words governed by social norms and learned behaviors).

Language doesn't need to be spoken only. In fact, almost all artifacts manufactured by humans are a form of language. A mug is a memory: its design is a memory of its purpose, and this is communicated visually. A bank is a memory for our money, and a way to communicate this money between different entities.

I think AI will emerge in a digital city, will become more performant with better memories, and AGI will take a life of its own with many AI agents that know how to build communicable artifacts.

A unified representation

While building higher and more appropriate representations (memory, language, etc.) is important to understand the mechanics of life, I think it is important to remember that these are words that we invented for our own convenience. Thus, one must be flexible to "ditch" these artificial boundaries when the time comes.

I think:

1. There is a single universal representation

2. Everything that we see is an emergent phenomena from:

   1. The parameterization of this representation (one instances)

   2. The interactions of multiple instances

As a simplified example, one can imagine the environment to be represented by 10-D (10 properties). Each agent in the environment in this case will also be a vector of 10-D. Each agent move is a matrix multiplication between its own 10-D and the environment's. The result is a change in both the environment and the agent.

The world is one big multi-dimensional tensor, and everything in this world is a world in itself, albeit very sparse, to represent our perception and capabilities to interact with the world.

So, how to proceed?

The key to success is:

1. Unlock fast iterations and feedback loops. The more the iterations, the better the results. The play will be on the long tail of events. Sheer amount of failures are required to unlock success.

2. Communication

   1. Writing

      1. Via daily writing, I hope to seed a positive feedback loop to set myself on the good path.

      2. Bi-weekly: every two weeks, I need to dedicate a day to aggregate all of these information in one place.

   2. Make youtube content on the bi-weekly basis (experiments, new learnings)

3. Build a community: While unfortunate that I will have to start this work in isolation, the medium to long term future will require new ideas and perspectives, that can only be unlocked by a community. To do this:

   1. Promote a discord server in all my communications

   2. Focus on high-quality communications. There is a double-benefit here. The higher the quality, the better my ideas are going to get. Also, it is a naturally "good-for-everyone" advertisement tool, to promote the community.

   3. Publish one-to-two articles only per year in Alife or GECCO conferences

      1. It adds credibility

      2. A key to access existing communities

4. Leverage LLMs. I need to build tools to achieve the following:

   1. Grokking through the papers

   2. Brainstorming

   3. Editing / proofreading articles