     .     AI, MCP  Harness
 


        ?   -      -.

  MCP (Model Context Protocol) -   (USB-C  Ȼ),       , API      .     ,     ,   Eval Harness -   ,       .

  :  ,    ,  (OAuth 2.1)   .       .

   ,       ,      .





 

     .     AI, MCP  Harness





 


.   -

 :     .

   -   10%   .

   : , , -.

 1. :   

 1.     (LLM)., , .           .

 2.    : RAG  .       (Retrieval-Augmented Generation)     (Function Calling).   .

 3.   -. ReAct (Reason + Act).   ,    ,  ,   .  :  (LLM),  (  ), .

:    Python,          .

 2.  :  

 4.  N?M.  10   10   (CRM,  , Slack,  )    .   API.

 5.   MCP (Model Context Protocol).

:MCP  USB-C  Ȼ.   Anthropic    Linux Foundation (Agentic AI Foundation)   2025 .

:Host, Client, Server.  (stdio     Streamable HTTP  ).

:Tools (), Resources (), Prompts ( ).

  Function Calling:MCP    ,            JSON-RPC 2.0.

:    3.           ,    MCP-  .        MCP-      .

 3.  :    MCP

 6.   MCP-. (handshake),  ,  ,  .

 7.     MCP. OAuth 2.1   2025 .          . Stateless- .

 8.  MCP.  .    (Google Drive, Slack, GitHub,  ERP).    MCP-.

:  .     MCP    Jira (), Confluence ( )   CRM,    .

 4.  : Harness  

 9.   .   .   Harness ()   .

 10.  Eval Harness.

  : ,    .

Rule-based checks:  ( JSON, regex,  ).

LLM-as-a-judge:     ,      .

 11. Harness Bench  .    (),          .  ,      .

: Harness  -.  ,   ,    unit-,       LLM-.

 5. :  

 12.   - ,   - .    .  .

 13.  .

 (Orchestrator) -  (Workers).

Peer-to-peer:   .

  :  MCP-   .      (,    )     MCP.

 14.      .  .    (blackboard).  .

:       .

-    MCP- .

-  ,    ( ) .

- (Orchestrator)     ,     MCP.

 6.   

 15.   . ,        .   MCP.

 16.  .     . .

 17. ,   .     ,       MCP.      .

.   

     .

 2026   :    (),  .



 (MCP Host, Harness, ReAct, LLM-as-a-judge).

-     .

    (MCP SDK, Inspect AI, Databricks Evaluation Framework).




.   -


 2022         .        ,      .   -  .     -   .     .

      . ,  ,       ,     ,    Jira         .    2022  2024 ,       .       ,             .

   .    ,     .

      ().         . - -     ,  ,    :    ,  ,    ,  ,      .      -      .

,        ,    .

  .     -:   ,   ,   .     :   PostgreSQL, CRM Salesforce,  , Slack, GitHub, ERP 1.    ,    ,   -  -     - .   20,   10,    200  .   API  Salesforce    .     .     .

  2024       .  Anthropic   Model Context Protocol (MCP)-  ,     USB-C   .

MCP      ,  - HTTP   .   ,  ,          .     200 .    MCP-       MCP-   .      -     OpenAI, Anthropic   LLM,     -        .   2025  Anthropic  MCP    Agentic AI Foundation   Linux Foundation,      .

    -    .          ( ,  ,  ),      .  ,         ?       ?  ,         ?

     Harness-  ,       ,    . Harness     ,      ,    ,      ,    -.

  -      ,    IT    .       .             ,       ,       MCP.

 ,    MCP   JSON-RPC,       OAuth 2.1,   Harness            ,        .

,      , .   ,  .  .




 1.     (LLM)


        ,    -,      ,     .    -    .    ,   .    LLM -   ,        .

        

     ,  ?,     ,    .   ,   -  ,       .     -    .

    .  -     .   (, Byte Pair Encoding)         .       : , , .   unbelievable  un, believ, able.

         ().      , : [1234, 5678, 90, 12].       .

:     

   LLM   (Transformer),   2017 .     -  (self-attention).

,    :     ,     .  ,     ,    .              .            .  ,              ,    .

      ,    - ,     .

 :    

    ,    .     (  +   ),       .

 2023     4 000   .  2025      128 000, 200 000   1  .

   .        ,      .  ,   :        ,           .            -        .

  

 -    .    .

     ,    -    .     .  ,           .

                 ,      ,   , .    ,     ,             .

   ,       ,  .      .   LLM   ,            (   MCP),   .

  

  ,         ,     .

 0:      .    ,         JSON-.          ,    .

  0 (, 0.7):     ,    .   , ,    .

  -  .      (         )       .

   -    ,           -  .             ,      . ,   ,   ,       -  .




 2.    : RAG  


      ,     -      ,      :        ?   ,       ,  ,    4.2  ,     ?

            .  2025         : Retrieval-Augmented Generation (RAG) Function Calling ( ).

RAG:    

RAG -  ,     ,     ,    .    LLM      . RAG      .

    ,   :

 ( ):    (PDF-, ,  Confluence,    ),       ()      (-)      -   ,   .         (Vector DB).

 (   ):

   ,        .

       ,         ( ).        :  ,        -      .

   ( )      .

         LLM.

 ,   ,   :  ,       .      - ,   .

 RAG:RAG     .   -   .    ,          (     ),             .

Function Calling:  

   ,   Function Calling( ).  ,   LLM     .

   :        JSON. : send_email(to, subject, body)  get_exchange_rate(currency).

        ,   ,  :

LLM    ,          ,    send_email.

      JSON-: {"name": "send_email", "arguments": {"to": "ivan@example.com", "subject": "", "body": ",  ..."}}.

   (,    )   JSON,        API  ,   (, success: true)    .

      : ,  .

 Function Calling:       ,     .     :    ,    SQL-,    Telegram,   .        .       -.       N?M,     .

 RAG  Function Calling -   

    RAG  Function Calling -   ,      .       .

,          ,  CRM   .

                Python-  .

         ,          .

       -    -  ,    .    ,      ,  -  ,       ,      ,       .

    ,   MCP   RAG-    ,    ,          ,      USB-.




 3.   -


  1  2    (LLM)        (RAG  Function Calling).        .        .

            .       -  - .      -  -  -  - .     ,    ,     .

 ReAct:  

        ReAct(Reason + Act -  + ).   ,         .

      , :   ,         100   ,       .     :

Reason ():        .       .        .

Act ():   get_sales_data(date: "2025-10-03").

Observation ():  JSON   .    .

Reason ():  ,   " X"    500 ,   .       .

Act ():  check_inventory(item: " X").

Observation ():  5000 ,    .

Act ():  reserve_stock(item: " X", amount: 100).

Final Answer ( ): .  100   X.

     .    6  ,     10 ,   :  .        ,    100  .    ,       ,    .

 : ,   

  ReAct    ,    .  - -   ,     :

 (LLM):.     Reason.    ,    (Observation)  ,     (Act).

 (Tools):  .       API,     .  :   ,     .        ,     ( ).

 (Memory):     ,       .      :

 :    . ,     .

  (Episodic):   .       ,  .

 :  ( ,    Pinecone  Chroma).     ,         ,      .

 (Planner):         ReAct    .   ,      510  ,    (  )        .    ,        .

: - 

   ,   Python,     :       .

 :            10.

  :

 1 (): (LLM)   .  ,      ,   - .

 2 ( 1):   search_wikipedia(query: "   ").

 3 ( 1):  :          .  1976      "-1".

 4 ():,    . ,   .    ,     ,     .

 5 ( ):      ,       ,     .   1976      "-1".

   :      (,   )      .   ReAct     .

   -  ,      ,             -         .

   ,        ,    (Slack, SQL, Google Drive, GitHub) ,     .          ,      API-  .     ,   MCP      ,     ,  .




 4.  N?M:  


       .    ,     .             .          ,  ,        .

   :  N?M.

 

   2025 .

   N - (). :

-,  .

-, 




  .


   .

   ,     (https://www.litres.ru/book/georgiy-dimitriadi/ot-izolirovannykh-modelei-k-mul-tiagentnym-ekosistemam-74572224/)  .

      Visa, MasterCard, Maestro,    ,   ,     ,  PayPal, WebMoney, ., QIWI ,       .


