Commit c3d0b8ef authored by Dimitrios Gogos's avatar Dimitrios Gogos
Browse files

docs: add overview diagrams

parent 3d177ea2

ai2/diagrams/overview.drawio.svg

0 → 100644
+40 −0
Changes for ai2/diagrams/overview.drawio.svg: 40 added lines, 0 removed lines.
Original line number Diff line number Diff line 
<svg xmlns="http://www.w3.org/2000/svg" version="1.1" width="1000px" height="520px" viewBox="0 0 1000 520" content="&lt;mxfile host=&quot;Electron&quot; type=&quot;device&quot;&gt;&lt;diagram id=&quot;d&quot; name=&quot;Page-1&quot;&gt;&lt;mxGraphModel dx=&quot;1000&quot; dy=&quot;520&quot; grid=&quot;1&quot; gridSize=&quot;10&quot; guides=&quot;1&quot; page=&quot;1&quot; pageWidth=&quot;1000&quot; pageHeight=&quot;520&quot; background=&quot;#ffffff&quot;&gt;&lt;root&gt;&lt;mxCell id=&quot;0&quot;/&gt;&lt;mxCell id=&quot;1&quot; parent=&quot;0&quot;/&gt;&lt;mxCell id=&quot;l1&quot; value=&quot;&amp;lt;b&amp;gt;Caller&amp;lt;/b&amp;gt;&quot; style=&quot;rounded=1;arcSize=4;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#9e9e9e;align=left;spacingLeft=12;fontSize=13;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;20&quot; y=&quot;20&quot; width=&quot;960&quot; height=&quot;80&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;l2&quot; value=&quot;&amp;lt;b&amp;gt;Agent&amp;lt;/b&amp;gt;&quot; style=&quot;rounded=1;arcSize=4;whiteSpace=wrap;html=1;fillColor=#e8f0fb;strokeColor=#6c8ebf;align=left;spacingLeft=12;fontSize=13;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;20&quot; y=&quot;140&quot; width=&quot;960&quot; height=&quot;100&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;l3&quot; value=&quot;&amp;lt;b&amp;gt;MCP server&amp;lt;/b&amp;gt;&quot; style=&quot;rounded=1;arcSize=4;whiteSpace=wrap;html=1;fillColor=#e8f0fb;strokeColor=#6c8ebf;align=left;spacingLeft=12;fontSize=13;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;20&quot; y=&quot;290&quot; width=&quot;960&quot; height=&quot;90&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;l4&quot; value=&quot;&amp;lt;b&amp;gt;OEG&amp;lt;/b&amp;gt;&quot; style=&quot;rounded=1;arcSize=4;whiteSpace=wrap;html=1;fillColor=#eaf5e6;strokeColor=#82b366;align=left;spacingLeft=12;fontSize=13;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;20&quot; y=&quot;420&quot; width=&quot;960&quot; height=&quot;80&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;portal&quot; value=&quot;&amp;lt;b&amp;gt;Portal backend&amp;lt;/b&amp;gt;&quot; style=&quot;rounded=1;arcSize=8;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#9e9e9e;fontSize=12;fontColor=#222222;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;300&quot; y=&quot;40&quot; width=&quot;220&quot; height=&quot;40&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;agent&quot; value=&quot;&amp;lt;b&amp;gt;agent (FastAPI)&amp;lt;/b&amp;gt;&amp;lt;br&amp;gt;ai2-ai-agent :9013&quot; style=&quot;rounded=1;arcSize=8;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#6c8ebf;fontSize=12;fontColor=#222222;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;300&quot; y=&quot;160&quot; width=&quot;220&quot; height=&quot;60&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;groq&quot; value=&quot;&amp;lt;b&amp;gt;Groq-hosted LLM&amp;lt;/b&amp;gt;&quot; style=&quot;rounded=1;arcSize=8;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#9e9e9e;fontSize=12;fontColor=#222222;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;700&quot; y=&quot;150&quot; width=&quot;200&quot; height=&quot;36&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;redis&quot; value=&quot;&amp;lt;b&amp;gt;Redis (optional)&amp;lt;/b&amp;gt;&quot; style=&quot;rounded=1;arcSize=8;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#9e9e9e;fontSize=12;fontColor=#222222;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;700&quot; y=&quot;196&quot; width=&quot;200&quot; height=&quot;36&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;mcp&quot; value=&quot;&amp;lt;b&amp;gt;MCP server (FastMCP)&amp;lt;/b&amp;gt;&amp;lt;br&amp;gt;ai2-mcp :8004 · 20 tools&quot; style=&quot;rounded=1;arcSize=8;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#6c8ebf;fontSize=12;fontColor=#222222;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;300&quot; y=&quot;305&quot; width=&quot;220&quot; height=&quot;60&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;oeg&quot; value=&quot;&amp;lt;b&amp;gt;OEG CAMARA APIs&amp;lt;/b&amp;gt;&amp;lt;br&amp;gt;checks the token's scopes&quot; style=&quot;rounded=1;arcSize=8;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#82b366;fontSize=12;fontColor=#222222;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;300&quot; y=&quot;435&quot; width=&quot;220&quot; height=&quot;50&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;e0&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;html=1;strokeColor=#555555;strokeWidth=1.3;endArrow=block;endFill=1;startArrow=none;startFill=1;exitX=0.5;exitY=1.0;exitDx=0;exitDy=0;entryX=0.5;entryY=0.0;entryDx=0;entryDy=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; source=&quot;portal&quot; target=&quot;agent&quot;&gt;&lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&lt;/mxGeometry&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;e1&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;html=1;strokeColor=#555555;strokeWidth=1.3;endArrow=block;endFill=1;startArrow=none;startFill=1;exitX=1.0;exitY=0.1333;exitDx=0;exitDy=0;entryX=0.0;entryY=0.5;entryDx=0;entryDy=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; source=&quot;agent&quot; target=&quot;groq&quot;&gt;&lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&lt;/mxGeometry&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;e2&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;html=1;strokeColor=#555555;strokeWidth=1.3;endArrow=block;endFill=1;startArrow=block;startFill=1;exitX=1.0;exitY=0.9;exitDx=0;exitDy=0;entryX=0.0;entryY=0.5;entryDx=0;entryDy=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; source=&quot;agent&quot; target=&quot;redis&quot;&gt;&lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&lt;/mxGeometry&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;e3&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;html=1;strokeColor=#555555;strokeWidth=1.3;endArrow=block;endFill=1;startArrow=none;startFill=1;exitX=0.5;exitY=1.0;exitDx=0;exitDy=0;entryX=0.5;entryY=0.0;entryDx=0;entryDy=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; source=&quot;agent&quot; target=&quot;mcp&quot;&gt;&lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&lt;/mxGeometry&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;e4&quot; style=&quot;edgeStyle=orthogonalEdgeStyle;rounded=0;html=1;strokeColor=#555555;strokeWidth=1.3;endArrow=block;endFill=1;startArrow=none;startFill=1;exitX=0.5;exitY=1.0;exitDx=0;exitDy=0;entryX=0.5;entryY=0.0;entryDx=0;entryDy=0;&quot; edge=&quot;1&quot; parent=&quot;1&quot; source=&quot;mcp&quot; target=&quot;oeg&quot;&gt;&lt;mxGeometry relative=&quot;1&quot; as=&quot;geometry&quot;&gt;&lt;/mxGeometry&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;t0&quot; value=&quot;POST /groq-mcp {query, session_id}&amp;lt;br&amp;gt;+ Bearer token&quot; style=&quot;text;html=1;align=left;verticalAlign=middle;fontSize=10.5;fontColor=#222222;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;416&quot; y=&quot;104&quot; width=&quot;260&quot; height=&quot;28&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;t1&quot; value=&quot;chat completions&quot; style=&quot;text;html=1;align=center;verticalAlign=middle;fontSize=10.5;fontColor=#222222;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;530&quot; y=&quot;152&quot; width=&quot;160&quot; height=&quot;14&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;t2&quot; value=&quot;recent turns&quot; style=&quot;text;html=1;align=center;verticalAlign=middle;fontSize=10.5;fontColor=#222222;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;530&quot; y=&quot;198&quot; width=&quot;160&quot; height=&quot;14&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;t3&quot; value=&quot;MCP over HTTP (/mcp),&amp;lt;br&amp;gt;same Bearer token&quot; style=&quot;text;html=1;align=left;verticalAlign=middle;fontSize=10.5;fontColor=#222222;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;416&quot; y=&quot;250&quot; width=&quot;200&quot; height=&quot;28&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;mxCell id=&quot;t4&quot; value=&quot;HTTPS/HTTP,&amp;lt;br&amp;gt;same Bearer token&quot; style=&quot;text;html=1;align=left;verticalAlign=middle;fontSize=10.5;fontColor=#222222;&quot; vertex=&quot;1&quot; parent=&quot;1&quot;&gt;&lt;mxGeometry x=&quot;416&quot; y=&quot;387&quot; width=&quot;200&quot; height=&quot;28&quot; as=&quot;geometry&quot;/&gt;&lt;/mxCell&gt;&lt;/root&gt;&lt;/mxGraphModel&gt;&lt;/diagram&gt;&lt;/mxfile&gt;">

<rect x="0" y="0" width="1000" height="520" fill="#ffffff"/>

<defs><marker id="ah" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path d="M0,0 L10,5 L0,10 z" fill="#555555"/></marker><marker id="am" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="5" markerHeight="5" orient="auto-start-reverse"><path d="M0,0 L10,5 L0,10 z" fill="#d79b00"/></marker></defs>

<rect x="20" y="20" width="960" height="80" rx="8" fill="#f5f5f5" stroke="#9e9e9e"/>

<text x="33" y="60.0" font-family="Helvetica, Arial, sans-serif" font-size="13" font-weight="bold" fill="#222222" text-anchor="start" dominant-baseline="central">Caller</text>

<rect x="20" y="140" width="960" height="100" rx="8" fill="#e8f0fb" stroke="#6c8ebf"/>

<text x="33" y="190.0" font-family="Helvetica, Arial, sans-serif" font-size="13" font-weight="bold" fill="#222222" text-anchor="start" dominant-baseline="central">Agent</text>

<rect x="20" y="290" width="960" height="90" rx="8" fill="#e8f0fb" stroke="#6c8ebf"/>

<text x="33" y="335.0" font-family="Helvetica, Arial, sans-serif" font-size="13" font-weight="bold" fill="#222222" text-anchor="start" dominant-baseline="central">MCP server</text>

<rect x="20" y="420" width="960" height="80" rx="8" fill="#eaf5e6" stroke="#82b366"/>

<text x="33" y="460.0" font-family="Helvetica, Arial, sans-serif" font-size="13" font-weight="bold" fill="#222222" text-anchor="start" dominant-baseline="central">OEG</text>

<rect x="300" y="40" width="220" height="40" rx="6" fill="#ffffff" stroke="#9e9e9e"/>

<text x="410.0" y="60.0" font-family="Helvetica, Arial, sans-serif" font-size="12" font-weight="bold" fill="#222222" text-anchor="middle" dominant-baseline="central">Portal backend</text>

<rect x="300" y="160" width="220" height="60" rx="6" fill="#ffffff" stroke="#6c8ebf"/>

<text x="410.0" y="182.5" font-family="Helvetica, Arial, sans-serif" font-size="12" font-weight="bold" fill="#222222" text-anchor="middle" dominant-baseline="central">agent (FastAPI)</text>

<text x="410.0" y="197.5" font-family="Helvetica, Arial, sans-serif" font-size="11" font-weight="normal" fill="#222222" text-anchor="middle" dominant-baseline="central">ai2-ai-agent :9013</text>

<rect x="700" y="150" width="200" height="36" rx="6" fill="#ffffff" stroke="#9e9e9e"/>

<text x="800.0" y="168.0" font-family="Helvetica, Arial, sans-serif" font-size="12" font-weight="bold" fill="#222222" text-anchor="middle" dominant-baseline="central">Groq-hosted LLM</text>

<rect x="700" y="196" width="200" height="36" rx="6" fill="#ffffff" stroke="#9e9e9e"/>

<text x="800.0" y="214.0" font-family="Helvetica, Arial, sans-serif" font-size="12" font-weight="bold" fill="#222222" text-anchor="middle" dominant-baseline="central">Redis (optional)</text>

<rect x="300" y="305" width="220" height="60" rx="6" fill="#ffffff" stroke="#6c8ebf"/>

<text x="410.0" y="327.5" font-family="Helvetica, Arial, sans-serif" font-size="12" font-weight="bold" fill="#222222" text-anchor="middle" dominant-baseline="central">MCP server (FastMCP)</text>

<text x="410.0" y="342.5" font-family="Helvetica, Arial, sans-serif" font-size="11" font-weight="normal" fill="#222222" text-anchor="middle" dominant-baseline="central">ai2-mcp :8004 · 20 tools</text>

<rect x="300" y="435" width="220" height="50" rx="6" fill="#ffffff" stroke="#82b366"/>

<text x="410.0" y="452.5" font-family="Helvetica, Arial, sans-serif" font-size="12" font-weight="bold" fill="#222222" text-anchor="middle" dominant-baseline="central">OEG CAMARA APIs</text>

<text x="410.0" y="467.5" font-family="Helvetica, Arial, sans-serif" font-size="11" font-weight="normal" fill="#222222" text-anchor="middle" dominant-baseline="central">checks the token's scopes</text>

<polyline points="410,80 410,160" fill="none" stroke="#555555" stroke-width="1.3" marker-end="url(#ah)"/>

<polyline points="520,168 700,168" fill="none" stroke="#555555" stroke-width="1.3" marker-end="url(#ah)"/>

<polyline points="520,214 700,214" fill="none" stroke="#555555" stroke-width="1.3" marker-end="url(#ah)" marker-start="url(#ah)"/>

<polyline points="410,220 410,305" fill="none" stroke="#555555" stroke-width="1.3" marker-end="url(#ah)"/>

<polyline points="410,365 410,435" fill="none" stroke="#555555" stroke-width="1.3" marker-end="url(#ah)"/>

<text x="416" y="112.0" font-family="Helvetica, Arial, sans-serif" font-size="10.5" font-weight="normal" fill="#222222" text-anchor="start" dominant-baseline="central">POST /groq-mcp {query, session_id}</text>

<text x="416" y="124.0" font-family="Helvetica, Arial, sans-serif" font-size="10.5" font-weight="normal" fill="#222222" text-anchor="start" dominant-baseline="central">+ Bearer token</text>

<text x="610.0" y="159.0" font-family="Helvetica, Arial, sans-serif" font-size="10.5" font-weight="normal" fill="#222222" text-anchor="middle" dominant-baseline="central">chat completions</text>

<text x="610.0" y="205.0" font-family="Helvetica, Arial, sans-serif" font-size="10.5" font-weight="normal" fill="#222222" text-anchor="middle" dominant-baseline="central">recent turns</text>

<text x="416" y="258.0" font-family="Helvetica, Arial, sans-serif" font-size="10.5" font-weight="normal" fill="#222222" text-anchor="start" dominant-baseline="central">MCP over HTTP (/mcp),</text>

<text x="416" y="270.0" font-family="Helvetica, Arial, sans-serif" font-size="10.5" font-weight="normal" fill="#222222" text-anchor="start" dominant-baseline="central">same Bearer token</text>

<text x="416" y="395.0" font-family="Helvetica, Arial, sans-serif" font-size="10.5" font-weight="normal" fill="#222222" text-anchor="start" dominant-baseline="central">HTTPS/HTTP,</text>

<text x="416" y="407.0" font-family="Helvetica, Arial, sans-serif" font-size="10.5" font-weight="normal" fill="#222222" text-anchor="start" dominant-baseline="central">same Bearer token</text>

</svg>

ai2/overview-and-architecture.md

+1 −22
Changes for ai2/overview-and-architecture.md: 1 added line, 22 removed lines.
Original line number Diff line number Diff line
@@ -36,28 +36,7 @@ tool call. OEG decides what the call may do. AI² holds no OEG credentials of it 
the last few turns of each conversation for a limited time. In the platform chart, nothing is

written to disk.



```text

  Portal backend  POST /groq-mcp  {query, session_id}  + Bearer token

        |

        v

  +----------------------+  chat completions   +-------------------+

  | agent  (FastAPI)     |-------------------->| Groq-hosted LLM   |

  | ai2-ai-agent :9013   |                     +-------------------+

  |                      |  recent turns       +-------------------+

  |                      |<------------------->| Redis (optional)  |

  +----------+-----------+                     +-------------------+

             |  MCP over HTTP (/mcp), same Bearer token

             v

  +----------------------+

  | MCP server (FastMCP) |  20 tools

  | ai2-mcp :8004        |

  +----------+-----------+

             |  HTTPS/HTTP, same Bearer token

             v

  +----------------------+

  | OEG CAMARA APIs      |  checks the token's scopes

  +----------------------+

```

![AI²: portal backend, agent, Groq, Redis, MCP server and OEG](diagrams/overview.drawio.svg)



---

data-lab/diagrams/overview.drawio.svg

0 → 100644
+77 −0

File added.

Preview size limit exceeded, changes collapsed.

data-lab/overview-and-architecture.md

+1 −23
Changes for data-lab/overview-and-architecture.md: 1 added line, 23 removed lines.
Original line number Diff line number Diff line
@@ -48,29 +48,7 @@ management console; both call the Data Lab API, which holds the hub API key and 
admin credential (who else may use the key: REQ-DL-11). piveau's own access control enforces who

may read and write each catalogue and dataset, whichever route a caller uses.



```text

   Contributors, consumers, stewards            Ingestion scripts

                  |                                     |

                  v                                     v

  +-------------------------------------------------------------------+

  | Platform Nginx (TLS)                       routes /data-lab/*     |

  +-------------------------------------------------------------------+

     | /data-lab/      | /data-lab/api/       | /data-lab/hub/  | /data-lab/search/

     v                 v                      v                 v

  +---------+ /api/ +---------------+    +--------------+   +--------------+

  | Data    |------>| Data Lab API  |--->| piveau       |-->| piveau       |

  | Lab UI  |       | holds hub key |    | hub-repo     |   | hub-search   |

  +---------+       | and KC admin  |--->| (write path) |   | (read path)  |

                    +---------------+    +--------------+   +--------------+

                       |                    |          |           |

                       v                    v          v           v

                 +------------+     +----------+ +----------+ +---------------+

                 | Keycloak   |     | Virtuoso | | shadow   | | Elasticsearch |

                 | dataspace  |     | records  | | Virtuoso | | index         |

                 +------------+     +----------+ +----------+ +---------------+



   Not in this release: EDC connector, S3 storage, DataOps / MLOps pipelines

```

![Data Lab: callers, platform Nginx, UI, API, piveau hub and stores](diagrams/overview.drawio.svg)



The API reads through both hub services; it writes only to `piveau-hub-repo`, which indexes each

write into `piveau-hub-search`.

fm/diagrams/overview.drawio.svg

0 → 100644
+60 −0

File added.

Preview size limit exceeded, changes collapsed.