Brain Architecture

Two theories collaboratively characterize the brain functioning - modularity and distributive processing. One of the keys to the success of artificial neural networks is the idea of distributive representation. However, the modularity and specialization property has not been well taken into account. I tend to think of a multi-way graph-structured architecture, with branching or mergining possibly at any point, an ultimate exention to skip connections, lateral connections and highway. Moreover, this gaint net would not be operated fully, but only a small fraction would be actived to form a computation subgraph. This implies a navigation problem that requires choosing nodes and edges at each step, a type of actions that happens in the computation graph. Finally, the computation graph may form a high-level conceptual or imagination world. I speculate that it might be attention that causes our consciouness flow. On the one hand, there must be an attention mechanism to draw out a subgraph and navigate the computation flow; on the other hand, attention can be seen as a mental action, consciously navigating us in our imagination world ...