fix content streaming

.gitignore

@@ -4,7 +4,7 @@ venv
 logs
 sqlStorage
 playground
-alembic.ini
 .DS_Store
 messages.pot
 activeConfig
+__pycache__

ARCHITECTURE.md

@@ -0,0 +1,275 @@
+# Обзор архитектуры системы
+
+Этот документ — единый и актуальный источник информации по платформе: архитектура, протоколы, данные, конфигурация, сценарии, эксплуатация. Заменяет собой разрозненные и устаревшие документы.
+
+## Содержание
+- Компоненты и топология
+- Децентрализованный слой (членство, оценка размера сети, репликации, метрики)
+- Загрузка и конвертация контента
+- Просмотр и покупка контента (UI/UX требования)
+- API (ключевые эндпойнты и полезная нагрузка)
+- Ключи и схемы данных (DHT)
+- Конфигурация и значения по умолчанию
+- Наблюдаемость и метрики
+- Диаграммы последовательностей (Mermaid)
+- Сборка и тестирование
+
+---
+
+## Компоненты и топология
+
+- Backend API: сервис на Sanic (Python) с бота́ми Telegram; база данных PostgreSQL (SQLAlchemy + Alembic).
+- Хранилище: локальная ФС (uploads/derivatives); IPFS (kubo) для ретривания/пининга; tusd (resumable upload).
+- Конвертеры: воркеры (ffmpeg) в контейнерах — `convert_v3`, `convert_process`.
+- Frontend: SPA (Vite + TypeScript), отдается nginx-контейнером.
+- Децентрализованный слой: встроенный DHT (в процессе) — членство, лизы реплик, метрики контента.
+
+```mermaid
+flowchart LR
+  Client -- TWA/HTTP --> Frontend
+  Frontend -- REST --> API[Backend API]
+  API -- tus hooks --> tusd
+  API -- SQL --> Postgres
+  API -- IPC --> Workers[Converters]
+  API -- IPFS --> IPFS
+  API -- DHT --> DHT[(In-Process DHT)]
+  DHT -- CRDT Merge --> DHT
+```
+
+---
+
+## Децентрализованный слой
+
+### Идентификаторы и версии
+- NodeID = blake3(Ed25519 публичного ключа) — шестнадцатеричная строка (256 бит).
+- ContentID = blake3(зашифрованного блоба) — неизменяемый идентификатор контента.
+- schema_version = v1 — фиксируется во всех DHT-ключах/записях.
+
+### Членство (membership)
+- Рукопожатие `/api/v1/network.handshake` — запрос подписан Ed25519; верифицируется на стороне получателя. Без корректной подписи — 400 BAD_SIGNATURE.
+- Полезная нагрузка включает: сведения о ноде (версия, возможности, IPFS), метрики, массив известных публичных нод, квитанции достижимости (reachability_receipts: issuer, target, ASN, timestamp, signature).
+- Состояние членства — CRDT LWW-Set (добавления/удаления) с TTL (`DHT_MEMBERSHIP_TTL=600` сек), плюс HyperLogLog для оценки мощности (N_local).
+- Фильтрация «островов»: ноды с `reachability_ratio < q` (по умолчанию `q=0.6`) исключаются при вычислении N_estimate и выборе реплик.
+- Итоговая оценка `N_estimate = max(валидных N_local от пиров)`.
+
+```mermaid
+sequenceDiagram
+  participant A as Узел A
+  participant B as Узел B
+  A->>B: POST /network.handshake {nonce, ts, node, receipts, signature}
+  B->>B: верификация ts/nonce, подписи
+  B->>B: upsert member; merge(receipts)
+  B-->>A: {node, known_public_nodes, n_estimate, server_signature}
+  A->>A: merge; N_estimate = max(N_local, полученные)
+```
+
+### Репликации и лизы
+- Выбор префикса: `p = max(0, round(log2(N_estimate / R_target)))`, где `R_target ≥ 3` (по умолчанию 3).
+- Ответственные ноды: чьи первые `p` бит NodeID совпадают с первыми `p` бит ContentID.
+- Лидер — минимальный NodeID среди ответственных.
+- Лидер выдаёт `replica_leases` (TTL=600 сек), соблюдая разнообразие: не менее 3 разных первых октетов IP и, если доступно, 3 разных ASN.
+- Ранжирование кандидатов — rendezvous score `blake3(ContentID || NodeID)`.
+- Сердцебиение (heartbeat) держателей — каждые 60 сек; 3 пропуска → признать down и переназначить ≤180 сек.
+- Недобор/перебор фиксируются в `conflict_log` и прометеус‑метриках.
+
+```mermaid
+stateDiagram-v2
+  [*] --> Discover
+  Discover: Рукопожатия + квитанции
+  Discover --> Active: TTL & кворм ASN
+  Active --> Leader: Выбор лидера префикса p
+  Leader --> Leased: Выдача лизов (diversity)
+  Leased --> Monitoring: Heartbeat 60s
+  Monitoring --> Reassign: 3 пропуска
+  Reassign --> Leased
+```
+
+### Метрики (окна)
+- На событии просмотра формируются дельты CRDT:
+  - PN‑Counter — количество просмотров;
+  - HyperLogLog — уникальные ViewID (ViewID = blake3(ContentID || соль_устройства));
+  - G‑Counter — watch_time, bytes_out, количество завершений.
+- Окно по часу (`DHT_METRIC_WINDOW_SEC`), ключ `MetricKey = blake3(ContentID || WindowID)`.
+- Мерджи коммутативные, детерминированные.
+
+---
+
+## Загрузка и конвертация контента
+
+1) Клиент грузит в `tusd` (resumable). Бэкенд получает HTTP‑hooks `/api/v1/upload.tus-hook`.
+2) Создается запись в БД для зашифрованного контента, воркеры размещают производные:
+   - для медиа — preview/low/high;
+   - для бинарей — оригинал (доступен только при наличии лицензии).
+3) `/api/v1/content.view` возвращает `display_options` и агрегированное состояние конвертации/загрузки.
+
+```mermaid
+sequenceDiagram
+  participant C as Клиент
+  participant T as tusd
+  participant B as Бэкенд
+  participant W as Воркеры
+  participant DB as PostgreSQL
+
+  C->>T: upload
+  T->>B: hooks (pre/post-finish)
+  B->>DB: create content
+  B->>W: очередь конвертации
+  W->>DB: derive/previews
+  C->>B: GET /content.view
+  B->>DB: resolve derivatives
+  B-->>C: display_options + status
+```
+
+---
+
+## Просмотр и покупка (UI/UX)
+
+- `/api/v1/content.view/<content_address>` определяет доступные отображения:
+  - бинарный контент без превью — оригинал только при наличии лицензии;
+  - аудио/видео — для неавторизованных preview/low, для имеющих доступ — decrypted_low/high.
+- В процессе конвертации фронтенд показывает статус «processing», без фальшивых ссылок.
+- Обложка (cover):
+  - фиксированный квадратный слот; изображение «вписывается» без растягивания/искажения;
+  - пустые области не заполняются чёрным — фон совпадает с фоном страницы.
+- Кнопки «Купить за TON/Stars»: всегда в одной строке (без горизонтального/вертикального скролла контента на малых экранах).
+
+```mermaid
+flowchart LR
+  View[content.view] --> Resolve[Определение деривативов]
+  Resolve --> Ready{Готово?}
+  Ready -- Нет --> Info[Статус: processing/pending]
+  Ready -- Да --> Options
+  Options -- Бинарь + нет лицензии --> HideOriginal[Скрыть оригинал]
+  Options -- Медиа + нет лицензии --> PreviewLow[preview/low]
+  Options -- Есть лицензия --> Decrypted[decrypted low/high|original]
+```
+
+---
+
+## API (ключевые)
+
+- `GET /api/system.version` — актуальность сервиса.
+- `POST /api/v1/network.handshake` — обмен членством (обязательная Ed25519‑подпись запроса). Пример запроса:
+
+```json
+{
+  "version": "3.0.0",
+  "schema_version": "v1",
+  "public_key": "<base58 ed25519 pubkey>",
+  "node_id": "<blake3(pubkey)>",
+  "public_host": "https://node.example",
+  "node_type": "public|private",
+  "metrics": {"uptime_sec": 123, "content_count": 42},
+  "capabilities": {"accepts_inbound": true, "is_bootstrap": false},
+  "ipfs": {"multiaddrs": ["/ip4/.../tcp/4001"], "peer_id": "..."},
+  "known_public_nodes": [],
+  "reachability_receipts": [],
+  "timestamp": 1710000000,
+  "nonce": "<hex>",
+  "signature": "<base58 ed25519 signature>"
+}
+```
+
+- `GET /api/v1/content.view/<content_address>` — `display_options`, `status`, `conversion`.
+- `GET /api/v1.5/storage/<file_hash>` — отдача файла.
+- `GET /metrics` — экспозиция метрик Prometheus (либо fallback‑дамп счётчиков).
+
+---
+
+## Ключи и схемы DHT
+
+- `MetaKey(content_id)` — метаданные репликаций:
+  - `replica_leases`: карта `{lease_id -> {node_id, issued_at, expires_at, asn, ip_first_octet, heartbeat_at, score}}`;
+  - `leader`: NodeID лидера; `revision`: номер ревизии;
+  - `conflict_log`: массив событий `UNDER/OVER/LEASE_EXPIRED` и т.п.
+
+- `MembershipKey(node_id)` — членство:
+  - `members`: LWW‑Set; `receipts`: LWW‑Set;
+  - `hll`: HyperLogLog; `reports`: карты локальных оценок N;
+  - `logical_counter`: логический счётчик для LWW‑доминации.
+
+- `MetricKey(content_id, window_id)` — метрики окна:
+  - `views`: PN‑Counter; `unique`: HLL; `watch_time`, `bytes_out`, `completions`: G‑Counters.
+
+Все записи подписываются и сливаются детерминированно: CRDT‑логика + LWW‑доминация (`logical_counter`, `timestamp`, `node_id`).
+
+---
+
+## Конфигурация и значения по умолчанию
+
+- Сеть/рукопожатия: `NODE_PRIVACY`, `PUBLIC_HOST`, `HANDSHAKE_INTERVAL_SEC`, `NETWORK_TLS_VERIFY`, IPFS‑пиры/бустрапы.
+- DHT:
+  - `DHT_MIN_RECEIPTS=5`, `DHT_MIN_REACHABILITY=0.6`, `DHT_MEMBERSHIP_TTL=600`;
+  - `DHT_REPLICATION_TARGET=3`, `DHT_LEASE_TTL=600`,
+  - `DHT_HEARTBEAT_INTERVAL=60`, `DHT_HEARTBEAT_MISS_THRESHOLD=3`;
+  - `DHT_MIN_ASN=3`, `DHT_MIN_IP_OCTETS=3`,
+  - `DHT_METRIC_WINDOW_SEC=3600`.
+- Конвертация: квоты `CONVERT_*`, `MAX_CONTENT_SIZE_MB`.
+
+Примечание: PoW‑допуски и Kademlia k‑buckets на текущем этапе не активированы в коде — заложены в дизайн и могут быть реализованы отдельно.
+
+---
+
+## Наблюдаемость и метрики
+
+Prometheus:
+- `dht_replication_under_total`, `dht_replication_over_total`, `dht_leader_changes_total`;
+- `dht_merge_conflicts_total`;
+- `dht_view_count_total`, `dht_unique_view_estimate`, `dht_watch_time_seconds`.
+
+Логи: структурированные ошибки HTTP (с id), `conflict_log` по репликациям, события регистрации нод.
+
+---
+
+## Диаграммы последовательностей (сводные)
+
+### Обновление N_estimate
+```mermaid
+sequenceDiagram
+  participant Peer
+  participant Membership
+  participant DHT
+  Peer->>Membership: handshake(payload, receipts)
+  Membership->>Membership: merge LWW/receipts
+  Membership->>Membership: update HLL и N_local
+  Membership->>DHT: persist MembershipKey
+  Membership->>Membership: N_estimate = max(valid reports)
+```
+
+### Выбор лидера и выдача лизов
+```mermaid
+sequenceDiagram
+  participant L as Leader
+  participant R as Responsible
+  L->>L: p = round(log2(N_est/R))
+  L->>R: rank by rendezvous(ContentID, NodeID)
+  L->>L: assign leases (diversity)
+  R-->>L: heartbeat/60s
+  L->>L: reassign on 3 misses
+```
+
+### Публикация метрик окна
+```mermaid
+sequenceDiagram
+  participant C as Client
+  participant API as Backend
+  participant M as Metrics
+  participant D as DHT
+  C->>API: GET content.view?watch_time,bytes_out
+  API->>M: record_view(delta)
+  M->>D: merge MetricKey(ContentID, window)
+  API-->>Prom: /metrics
+```
+
+---
+
+## Сборка и тестирование
+
+```bash
+# Старт окружения (пример для /home/configs)
+docker compose -f /home/configs/docker-compose.yml --env-file /home/configs/.env up -d --build
+
+# Тесты слоя DHT
+cd uploader-bot
+python3 -m unittest discover -s tests/dht
+```

README.md

@@ -1,5 +1,14 @@
 # Sanic Telegram Bot [template]
 
+Полная документация по системе (архитектура, протоколы, конфигурация, диаграммы) — см. `ARCHITECTURE.md`.
+
+### Запуск тестов интеграции DHT
+
+```shell
+cd uploader-bot
+python3 -m unittest discover -s tests/dht
+```
+
 ---
 ## Run
 ```shell

alembic.ini

@@ -0,0 +1,35 @@
+[alembic]
+script_location = alembic
+sqlalchemy.url = ${DATABASE_URL}
+
+[loggers]
+keys = root,sqlalchemy,alembic
+
+[handlers]
+keys = console
+
+[formatters]
+keys = generic
+
+[logger_root]
+level = WARN
+handlers = console
+
+[logger_sqlalchemy]
+level = WARN
+handlers =
+qualname = sqlalchemy.engine
+
+[logger_alembic]
+level = INFO
+handlers =
+qualname = alembic
+
+[handler_console]
+class = StreamHandler
+args = (sys.stderr,)
+level = NOTSET
+formatter = generic
+
+[formatter_generic]
+format = %(levelname)-5.5s [%(name)s] %(message)s

alembic/env.py

@@ -1,3 +1,4 @@
+import os
 from logging.config import fileConfig
 
 from sqlalchemy import engine_from_config
@@ -7,6 +8,10 @@ from alembic import context
 
 config = context.config
 
+database_url = os.environ.get("DATABASE_URL")
+if database_url:
+    config.set_main_option("sqlalchemy.url", database_url)
+
 if config.config_file_name is not None:
     fileConfig(config.config_file_name)
 

alembic/versions/b1f2d3c4a5b6_add_artist_to_encrypted_content.py

@@ -0,0 +1,26 @@
+"""add artist column to encrypted content
+
+Revision ID: b1f2d3c4a5b6
+Revises: a7c1357e8d15
+Create Date: 2024-06-05 00:00:00.000000
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+
+
+# revision identifiers, used by Alembic.
+revision: str = 'b1f2d3c4a5b6'
+down_revision: Union[str, None] = 'a7c1357e8d15'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column('encrypted_contents', sa.Column('artist', sa.String(length=512), nullable=True))
+
+
+def downgrade() -> None:
+    op.drop_column('encrypted_contents', 'artist')

alembic/versions/c2d4e6f8a1b2_expand_telegram_id_precision.py

@@ -0,0 +1,38 @@
+"""expand telegram_id precision on stars invoices
+
+Revision ID: c2d4e6f8a1b2
+Revises: b1f2d3c4a5b6
+Create Date: 2025-10-17 00:00:00.000000
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+
+
+# revision identifiers, used by Alembic.
+revision: str = 'c2d4e6f8a1b2'
+down_revision: Union[str, None] = 'b1f2d3c4a5b6'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.alter_column(
+        'stars_invoices',
+        'telegram_id',
+        existing_type=sa.Integer(),
+        type_=sa.BigInteger(),
+        existing_nullable=True,
+    )
+
+
+def downgrade() -> None:
+    op.alter_column(
+        'stars_invoices',
+        'telegram_id',
+        existing_type=sa.BigInteger(),
+        type_=sa.Integer(),
+        existing_nullable=True,
+    )

alembic/versions/d3e5f7a9c0d1_create_dht_and_rdap_tables.py

@@ -0,0 +1,70 @@
+"""create dht_records and rdap_cache tables
+
+Revision ID: d3e5f7a9c0d1
+Revises: c2d4e6f8a1b2
+Create Date: 2025-10-22 00:00:00.000000
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+
+
+# revision identifiers, used by Alembic.
+revision: str = 'd3e5f7a9c0d1'
+down_revision: Union[str, None] = 'c2d4e6f8a1b2'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    bind = op.get_bind()
+    inspector = sa.inspect(bind)
+
+    # dht_records
+    if not inspector.has_table('dht_records'):
+        op.create_table(
+            'dht_records',
+            sa.Column('fingerprint', sa.String(length=128), primary_key=True),
+            sa.Column('key', sa.String(length=512), nullable=False),
+            sa.Column('schema_version', sa.String(length=16), nullable=False, server_default='v1'),
+            sa.Column('logical_counter', sa.Integer(), nullable=False, server_default='0'),
+            sa.Column('timestamp', sa.Float(), nullable=False, server_default='0'),
+            sa.Column('node_id', sa.String(length=128), nullable=False),
+            sa.Column('signature', sa.String(length=512), nullable=True),
+            sa.Column('value', sa.JSON(), nullable=False, server_default=sa.text("'{}'::jsonb")),
+            sa.Column('updated_at', sa.DateTime(), nullable=False, server_default=sa.text('CURRENT_TIMESTAMP')),
+        )
+    # ensure index exists (but don't fail if it already exists)
+    try:
+        existing_indexes = {idx['name'] for idx in inspector.get_indexes('dht_records')}
+    except Exception:
+        existing_indexes = set()
+    if 'ix_dht_records_key' not in existing_indexes:
+        op.create_index('ix_dht_records_key', 'dht_records', ['key'])
+
+    # rdap_cache
+    if not inspector.has_table('rdap_cache'):
+        op.create_table(
+            'rdap_cache',
+            sa.Column('ip', sa.String(length=64), primary_key=True),
+            sa.Column('asn', sa.Integer(), nullable=True),
+            sa.Column('source', sa.String(length=64), nullable=True),
+            sa.Column('updated_at', sa.DateTime(), nullable=False, server_default=sa.text('CURRENT_TIMESTAMP')),
+        )
+
+
+def downgrade() -> None:
+    try:
+        op.drop_table('rdap_cache')
+    except Exception:
+        pass
+    try:
+        op.drop_index('ix_dht_records_key', table_name='dht_records')
+    except Exception:
+        pass
+    try:
+        op.drop_table('dht_records')
+    except Exception:
+        pass

app/__main__.py

@@ -25,7 +25,32 @@ if int(os.getenv("SANIC_MAINTENANCE", '0')) == 1:
     while True:
         time.sleep(1)
 
-from app.core.models import Memory
+
+def init_db_schema_sync() -> None:
+    """Initialise all SQLAlchemy models in the database before services start.
+
+    This ensures that every table defined on AlchemyBase.metadata (including
+    newer ones like DHT and service_config) exists before any component
+    accesses the database.
+    """
+    try:
+        from sqlalchemy import create_engine
+        from app.core.models import AlchemyBase  # imports all models and populates metadata
+
+        db_url = os.environ.get('DATABASE_URL')
+        if not db_url:
+            raise RuntimeError('DATABASE_URL is not set')
+
+        # Normalise DSN to sync driver for schema creation
+        if '+asyncpg' in db_url:
+            db_url_sync = db_url.replace('+asyncpg', '+psycopg2')
+        else:
+            db_url_sync = db_url
+
+        sync_engine = create_engine(db_url_sync, pool_pre_ping=True)
+        AlchemyBase.metadata.create_all(sync_engine)
+    except Exception as e:
+        make_log('Startup', f'DB sync init failed: {e}', level='error')
 
 
 async def queue_daemon(app):
@@ -78,37 +103,22 @@ async def execute_queue(app):
 
 
 if __name__ == '__main__':
+    # Ensure DB schema is fully initialised for all models
+    init_db_schema_sync()
+
+    from app.core.models import Memory
     main_memory = Memory()
     if startup_target == '__main__':
         # Defer heavy imports to avoid side effects in background services
         # Mark this process as the primary node for seeding/config init
         os.environ.setdefault('NODE_ROLE', 'primary')
-        # Create DB tables synchronously before importing HTTP app to satisfy _secrets
-        try:
-            from sqlalchemy import create_engine
-            from app.core.models import AlchemyBase  # imports all models
-            db_url = os.environ.get('DATABASE_URL')
-            if not db_url:
-                raise RuntimeError('DATABASE_URL is not set')
-            # Normalize to sync driver
-            if '+asyncpg' in db_url:
-                db_url_sync = db_url.replace('+asyncpg', '+psycopg2')
-            else:
-                db_url_sync = db_url
-            sync_engine = create_engine(db_url_sync, pool_pre_ping=True)
-            AlchemyBase.metadata.create_all(sync_engine)
-        except Exception as e:
-            make_log('Startup', f'DB sync init failed: {e}', level='error')
         from app.api import app
-        from app.bot import dp as uploader_bot_dp
-        from app.client_bot import dp as client_bot_dp
+        # Delay aiogram dispatcher creation until loop is running
         from app.core._config import SANIC_PORT, PROJECT_HOST, DATABASE_URL
         from app.core.network.nodes import network_handshake_daemon, bootstrap_once_and_exit_if_failed
+        from app.core.network.maintenance import replication_daemon, heartbeat_daemon, dht_gossip_daemon
 
         app.ctx.memory = main_memory
-        for _target in [uploader_bot_dp, client_bot_dp]:
-            _target._s_memory = app.ctx.memory
-
         app.ctx.memory._app = app
 
         # Ensure DB schema exists using the same event loop as Sanic (idempotent)
@@ -116,11 +126,28 @@ if __name__ == '__main__':
 
         app.add_task(execute_queue(app))
         app.add_task(queue_daemon(app))
-        app.add_task(uploader_bot_dp.start_polling(app.ctx.memory._telegram_bot))
-        app.add_task(client_bot_dp.start_polling(app.ctx.memory._client_telegram_bot))
+        # Start bots after loop is ready
+        async def _start_bots():
+            try:
+                from app.bot import create_dispatcher as create_uploader_dp
+                from app.client_bot import create_dispatcher as create_client_dp
+                uploader_bot_dp = create_uploader_dp()
+                client_bot_dp = create_client_dp()
+                for _target in [uploader_bot_dp, client_bot_dp]:
+                    _target._s_memory = app.ctx.memory
+                await asyncio.gather(
+                    uploader_bot_dp.start_polling(app.ctx.memory._telegram_bot),
+                    client_bot_dp.start_polling(app.ctx.memory._client_telegram_bot),
+                )
+            except Exception as e:
+                make_log('Bots', f'Failed to start bots: {e}', level='error')
+        app.add_task(_start_bots())
         # Start network handshake daemon and bootstrap step
         app.add_task(network_handshake_daemon(app))
         app.add_task(bootstrap_once_and_exit_if_failed())
+        app.add_task(replication_daemon(app))
+        app.add_task(heartbeat_daemon(app))
+        app.add_task(dht_gossip_daemon(app))
 
         app.run(host='0.0.0.0', port=SANIC_PORT)
     else:
@@ -151,6 +178,9 @@ if __name__ == '__main__':
         elif startup_target == 'derivative_janitor':
             from app.core.background.derivative_cache_janitor import main_fn as target_fn
             time.sleep(5)
+        elif startup_target == 'events_sync':
+            from app.core.background.event_sync_service import main_fn as target_fn
+            time.sleep(5)
 
         startup_fn = startup_fn or target_fn
         assert startup_fn
@@ -163,7 +193,11 @@ if __name__ == '__main__':
                          level='error')
                 sys.exit(1)
 
-        loop = asyncio.get_event_loop()
+        try:
+            loop = asyncio.get_event_loop()
+        except RuntimeError:
+            loop = asyncio.new_event_loop()
+            asyncio.set_event_loop(loop)
         try:
             # Background services no longer perform schema initialization
             loop.run_until_complete(wrapped_startup_fn(main_memory))

app/api/__init__.py

@@ -20,11 +20,12 @@ from app.api.routes.network import (
     s_api_v1_network_nodes,
     s_api_v1_network_handshake,
 )
+from app.api.routes.network_events import s_api_v1_network_events
 from app.api.routes.auth import s_api_v1_auth_twa, s_api_v1_auth_select_wallet, s_api_v1_auth_me
 from app.api.routes.statics import s_api_tonconnect_manifest, s_api_platform_metadata
 from app.api.routes.node_storage import s_api_v1_storage_post, s_api_v1_storage_get, \
     s_api_v1_storage_decode_cid
-from app.api.routes.progressive_storage import s_api_v1_5_storage_get, s_api_v1_5_storage_post
+from app.api.routes.progressive_storage import s_api_v1_5_storage_get, s_api_v1_5_storage_post, s_api_v1_storage_fetch, s_api_v1_storage_proxy
 from app.api.routes.upload_tus import s_api_v1_upload_tus_hook
 from app.api.routes.account import s_api_v1_account_get
 from app.api.routes._blockchain import s_api_v1_blockchain_send_new_content_message, \
@@ -36,9 +37,11 @@ from app.api.routes.admin import (
     s_api_v1_admin_blockchain,
     s_api_v1_admin_cache_cleanup,
     s_api_v1_admin_cache_setlimits,
+    s_api_v1_admin_events,
     s_api_v1_admin_licenses,
     s_api_v1_admin_login,
     s_api_v1_admin_logout,
+    s_api_v1_admin_users_setadmin,
     s_api_v1_admin_node_setrole,
     s_api_v1_admin_nodes,
     s_api_v1_admin_overview,
@@ -49,11 +52,16 @@ from app.api.routes.admin import (
     s_api_v1_admin_system,
     s_api_v1_admin_uploads,
     s_api_v1_admin_users,
+    s_api_v1_admin_network,
+    s_api_v1_admin_network_config,
+    s_api_v1_admin_network_config_set,
 )
 from app.api.routes.tonconnect import s_api_v1_tonconnect_new, s_api_v1_tonconnect_logout
 from app.api.routes.keys import s_api_v1_keys_request
 from app.api.routes.sync import s_api_v1_sync_pin, s_api_v1_sync_status
 from app.api.routes.upload_status import s_api_v1_upload_status
+from app.api.routes.metrics import s_api_metrics
+from app.api.routes.dht import s_api_v1_dht_get, s_api_v1_dht_put
 
 
 app.add_route(s_index, "/", methods=["GET", "OPTIONS"])
@@ -66,6 +74,7 @@ app.add_route(s_api_system_send_status, "/api/system.sendStatus", methods=["POST
 app.add_route(s_api_v1_network_info, "/api/v1/network.info", methods=["GET", "OPTIONS"])
 app.add_route(s_api_v1_network_nodes, "/api/v1/network.nodes", methods=["GET", "OPTIONS"])
 app.add_route(s_api_v1_network_handshake, "/api/v1/network.handshake", methods=["POST", "OPTIONS"])
+app.add_route(s_api_v1_network_events, "/api/v1/network.events", methods=["GET", "OPTIONS"])
 
 app.add_route(s_api_tonconnect_manifest, "/api/tonconnect-manifest.json", methods=["GET", "OPTIONS"])
 app.add_route(s_api_platform_metadata, "/api/platform-metadata.json", methods=["GET", "OPTIONS"])
@@ -79,6 +88,8 @@ app.add_route(s_api_v1_tonconnect_logout, "/api/v1/tonconnect.logout", methods=[
 
 app.add_route(s_api_v1_5_storage_post, "/api/v1.5/storage", methods=["POST", "OPTIONS"])
 app.add_route(s_api_v1_5_storage_get, "/api/v1.5/storage/<file_hash>", methods=["GET", "OPTIONS"])
+app.add_route(s_api_v1_storage_fetch, "/api/v1/storage.fetch/<file_hash>", methods=["GET", "OPTIONS"])
+app.add_route(s_api_v1_storage_proxy, "/api/v1/storage.proxy/<file_hash>", methods=["GET", "OPTIONS"])
 
 app.add_route(s_api_v1_storage_post, "/api/v1/storage", methods=["POST", "OPTIONS"])
 app.add_route(s_api_v1_storage_get, "/api/v1/storage/<file_hash>", methods=["GET", "OPTIONS"])
@@ -102,8 +113,10 @@ app.add_route(s_api_v1_admin_overview, "/api/v1/admin.overview", methods=["GET",
 app.add_route(s_api_v1_admin_storage, "/api/v1/admin.storage", methods=["GET", "OPTIONS"])
 app.add_route(s_api_v1_admin_uploads, "/api/v1/admin.uploads", methods=["GET", "OPTIONS"])
 app.add_route(s_api_v1_admin_users, "/api/v1/admin.users", methods=["GET", "OPTIONS"])
+app.add_route(s_api_v1_admin_users_setadmin, "/api/v1/admin.users.setAdmin", methods=["POST", "OPTIONS"])
 app.add_route(s_api_v1_admin_licenses, "/api/v1/admin.licenses", methods=["GET", "OPTIONS"])
 app.add_route(s_api_v1_admin_stars, "/api/v1/admin.stars", methods=["GET", "OPTIONS"])
+app.add_route(s_api_v1_admin_events, "/api/v1/admin.events", methods=["GET", "OPTIONS"])
 app.add_route(s_api_v1_admin_system, "/api/v1/admin.system", methods=["GET", "OPTIONS"])
 app.add_route(s_api_v1_admin_blockchain, "/api/v1/admin.blockchain", methods=["GET", "OPTIONS"])
 app.add_route(s_api_v1_admin_node_setrole, "/api/v1/admin.node.setRole", methods=["POST", "OPTIONS"])
@@ -112,6 +125,9 @@ app.add_route(s_api_v1_admin_status, "/api/v1/admin.status", methods=["GET", "OP
 app.add_route(s_api_v1_admin_cache_setlimits, "/api/v1/admin.cache.setLimits", methods=["POST", "OPTIONS"])
 app.add_route(s_api_v1_admin_cache_cleanup, "/api/v1/admin.cache.cleanup", methods=["POST", "OPTIONS"])
 app.add_route(s_api_v1_admin_sync_setlimits, "/api/v1/admin.sync.setLimits", methods=["POST", "OPTIONS"])
+app.add_route(s_api_v1_admin_network, "/api/v1/admin.network", methods=["GET", "OPTIONS"])
+app.add_route(s_api_v1_admin_network_config, "/api/v1/admin.network.config", methods=["GET", "OPTIONS"])
+app.add_route(s_api_v1_admin_network_config_set, "/api/v1/admin.network.config.set", methods=["POST", "OPTIONS"])
 
 # tusd HTTP hooks
 app.add_route(s_api_v1_upload_tus_hook, "/api/v1/upload.tus-hook", methods=["POST", "OPTIONS"])
@@ -121,6 +137,9 @@ app.add_route(s_api_v1_keys_request, "/api/v1/keys.request", methods=["POST", "O
 app.add_route(s_api_v1_sync_pin, "/api/v1/sync.pin", methods=["POST", "OPTIONS"])
 app.add_route(s_api_v1_sync_status, "/api/v1/sync.status", methods=["GET", "OPTIONS"])
 app.add_route(s_api_v1_upload_status, "/api/v1/upload.status/<upload_id>", methods=["GET", "OPTIONS"])
+app.add_route(s_api_metrics, "/metrics", methods=["GET", "OPTIONS"])
+app.add_route(s_api_v1_dht_get, "/api/v1/dht.get", methods=["GET", "OPTIONS"])
+app.add_route(s_api_v1_dht_put, "/api/v1/dht.put", methods=["POST", "OPTIONS"])
 
 
 @app.exception(BaseException)

app/api/routes/_blockchain.py

@@ -56,6 +56,15 @@ async def s_api_v1_blockchain_send_new_content_message(request):
             assert field_key in request.json, f"No {field_key} provided"
             assert field_value(request.json[field_key]), f"Invalid {field_key} provided"
 
+        artist = request.json.get('artist')
+        if artist is not None:
+            assert isinstance(artist, str), "Invalid artist provided"
+            artist = artist.strip()
+            if artist == "":
+                artist = None
+        else:
+            artist = None
+
         # Support legacy: 'content' as decrypted ContentId; and new: 'content' as encrypted IPFS CID
         source_content_cid, cid_err = resolve_content(request.json['content'])
         assert not cid_err, f"Invalid content CID provided: {cid_err}"
@@ -85,11 +94,16 @@ async def s_api_v1_blockchain_send_new_content_message(request):
             image_content = None
 
 
-        content_title = f"{', '.join(request.json['authors'])} – {request.json['title']}" if request.json['authors'] else request.json['title']
+        content_title = request.json['title']
+        if artist:
+            content_title = f"{artist} – {content_title}"
+        elif request.json['authors']:
+            content_title = f"{', '.join(request.json['authors'])} – {request.json['title']}"
 
         metadata_content = await create_metadata_for_item(
             request.ctx.db_session,
-            title=content_title,
+            title=request.json['title'],
+            artist=artist,
             cover_url=f"{PROJECT_HOST}/api/v1.5/storage/{image_content_cid.serialize_v2()}" if image_content_cid else None,
             authors=request.json['authors'],
             hashtags=request.json['hashtags'],

app/api/routes/admin.py

@@ -7,11 +7,12 @@ import shutil
 from collections import defaultdict
 from datetime import datetime
 from pathlib import Path
-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, List, Optional, Tuple
+from urllib.parse import urlparse
 
-from base58 import b58encode
+from app.core._utils.b58 import b58encode
 from sanic import response
-from sqlalchemy import Integer, String, and_, case, cast, func, or_, select
+from sqlalchemy import Integer, String, and_, case, cast, func, or_, select, Text
 
 from app.api.routes._system import get_git_info
 from app.core._blockchain.ton.platform import platform
@@ -20,6 +21,7 @@ from app.core._config import (
     BACKEND_LOGS_DIR_HOST,
     LOG_DIR,
     CLIENT_TELEGRAM_BOT_USERNAME,
+    TELEGRAM_BOT_USERNAME,
     PROJECT_HOST,
     UPLOADS_DIR,
 )
@@ -34,7 +36,8 @@ from app.core.models.content_v3 import (
     IpfsSync,
     UploadSession,
 )
-from app.core.models.my_network import KnownNode
+from app.core.models.my_network import KnownNode, RemoteContentIndex
+from app.core.models.events import NodeEvent
 from app.core.models.tasks import BlockchainTask
 from app.core.models.node_storage import StoredContent
 from app.core.models.user import User
@@ -44,6 +47,10 @@ from app.core.models.wallet_connection import WalletConnection
 from app.core.models.user_activity import UserActivity
 from app.core._utils.share_links import build_content_links
 from app.core.content.content_id import ContentId
+from app.core.events.service import record_event
+from app.core.network.dht import MetricKey  # type stub; used in typing only
+from app.core.network.dht import dht_config
+from app.core.models._config import ServiceConfig
 
 MIN_ONCHAIN_INDEX = int(os.getenv("MIN_ONCHAIN_INDEX", "8"))
 
@@ -161,6 +168,38 @@ def _service_states(request) -> List[Dict[str, Any]]:
     return items
 
 
+def _node_public_base(node: KnownNode) -> Optional[str]:
+    meta = node.meta or {}
+    public_host = (meta.get('public_host') or '').strip()
+    if public_host:
+        base = public_host.rstrip('/')
+        if base.startswith('http://') or base.startswith('https://'):
+            return base
+        scheme = 'https' if node.port == 443 else 'http'
+        return f"{scheme}://{base.lstrip('/')}"
+    scheme = 'https' if node.port == 443 else 'http'
+    host = (node.ip or '').strip()
+    if not host:
+        return None
+    default_port = 443 if scheme == 'https' else 80
+    if node.port and node.port != default_port:
+        return f"{scheme}://{host}:{node.port}"
+    return f"{scheme}://{host}"
+
+
+def _node_gateway_base(node: KnownNode) -> Optional[str]:
+    meta = node.meta or {}
+    public_host = meta.get('public_host') or node.ip or ''
+    if not public_host:
+        return None
+    parsed = urlparse(public_host if '://' in public_host else f"https://{public_host}")
+    hostname = parsed.hostname or (node.ip or '').strip()
+    if not hostname:
+        return None
+    port = parsed.port or 8080
+    return f"http://{hostname}:{port}"
+
+
 def _format_dt(value: Optional[datetime]) -> Optional[str]:
     return value.isoformat() + 'Z' if isinstance(value, datetime) else None
 
@@ -181,11 +220,14 @@ def _storage_download_url(file_hash: Optional[str]) -> Optional[str]:
 def _pick_primary_download(candidates: List[tuple[str, Optional[str], Optional[int]]]) -> Optional[str]:
     priority = (
         'decrypted_high',
+        'decrypted_original',
         'decrypted_low',
         'decrypted_preview',
         'high',
         'low',
         'preview',
+        'original',
+        'stored',
     )
     for target in priority:
         for kind, url, _ in candidates:
@@ -269,6 +311,19 @@ async def s_api_v1_admin_overview(request):
 
     node_id = b58encode(hot_pubkey).decode()
 
+    highload_address: Optional[str] = None
+    try:
+        from tonsdk.contract.wallet import Wallets
+
+        highload_wallet = Wallets.ALL['hv3'](
+            private_key=service_wallet.options['private_key'],
+            public_key=service_wallet.options['public_key'],
+            wc=0,
+        )
+        highload_address = highload_wallet.address.to_string(1, 1, 1)
+    except Exception as exc:  # pragma: no cover
+        make_log("Admin", f"Failed to compute highload wallet address: {exc}", level="error")
+
     overview_payload = {
         'project': {
             'host': PROJECT_HOST,
@@ -283,6 +338,7 @@ async def s_api_v1_admin_overview(request):
             'id': node_id,
             'service_wallet': service_wallet.address.to_string(1, 1, 1),
             'ton_master': platform.address.to_string(1, 1, 1),
+            'highload_wallet': highload_address,
         },
         'runtime': {
             'python': py_platform.python_version(),
@@ -478,6 +534,18 @@ async def s_api_v1_admin_uploads(request):
         for content_id, count in license_rows:
             license_counts[int(content_id)] = int(count)
 
+    remote_map: Dict[str, List[Tuple[RemoteContentIndex, KnownNode]]] = defaultdict(list)
+    if encrypted_cids:
+        remote_rows = (await session.execute(
+            select(RemoteContentIndex, KnownNode)
+            .join(KnownNode, RemoteContentIndex.remote_node_id == KnownNode.id)
+            .where(RemoteContentIndex.encrypted_hash.in_(encrypted_cids))
+        )).all()
+        for remote_row, node in remote_rows:
+            if not remote_row.encrypted_hash:
+                continue
+            remote_map[remote_row.encrypted_hash].append((remote_row, node))
+
     contents_payload: List[Dict[str, Any]] = []
     category_totals: Dict[str, int] = {key: 0 for key in ALLOWED_UPLOAD_FILTERS if key != 'all'}
     matched_total = 0
@@ -599,6 +667,66 @@ async def s_api_v1_admin_uploads(request):
             if url
         ]
 
+        distribution_nodes: List[Dict[str, Any]] = []
+        meta_local_host = urlparse(PROJECT_HOST) if PROJECT_HOST else None
+        if stored:
+            distribution_nodes.append({
+                'node_id': None,
+                'is_local': True,
+                'host': (meta_local_host.hostname if meta_local_host and meta_local_host.hostname else 'local'),
+                'public_host': PROJECT_HOST.rstrip('/') if PROJECT_HOST else None,
+                'version': None,
+                'role': 'self',
+                'last_seen': None,
+                'content': {
+                    'encrypted_cid': content.encrypted_cid,
+                    'content_type': content.content_type,
+                    'size_bytes': content.enc_size_bytes,
+                    'preview_enabled': content.preview_enabled,
+                    'updated_at': _format_dt(content.updated_at),
+                    'metadata_cid': metadata_cid,
+                    'issuer_node_id': None,
+                },
+                'links': {
+                    'web_view': web_view_url,
+                    'api_view': f"{PROJECT_HOST}/api/v1/content.view/{share_target}" if PROJECT_HOST else None,
+                    'gateway_view': None,
+                },
+            })
+
+        remote_entries = remote_map.get(content.encrypted_cid, [])
+        for remote_row, node in remote_entries:
+            node_meta = node.meta or {}
+            base_url = _node_public_base(node)
+            gateway_base = _node_gateway_base(node)
+            remote_meta = remote_row.meta if isinstance(remote_row.meta, dict) else {}
+            remote_share_target = remote_meta.get('share_target') or content.encrypted_cid
+            distribution_nodes.append({
+                'node_id': node.id,
+                'is_local': False,
+                'host': node.ip,
+                'public_host': node_meta.get('public_host'),
+                'version': node_meta.get('version'),
+                'role': node_meta.get('role') or 'read-only',
+                'last_seen': _format_dt(node.last_sync),
+                'content': {
+                    'encrypted_cid': remote_row.encrypted_hash,
+                    'content_type': remote_row.content_type,
+                    'size_bytes': remote_meta.get('size_bytes'),
+                    'preview_enabled': remote_meta.get('preview_enabled'),
+                    'updated_at': _format_dt(remote_row.last_updated),
+                    'metadata_cid': remote_meta.get('metadata_cid'),
+                    'issuer_node_id': remote_meta.get('issuer_node_id'),
+                },
+                'links': {
+                    'web_view': f"{base_url}/viewContent?content={remote_share_target}" if base_url else None,
+                    'api_view': f"{base_url}/api/v1/content.view/{remote_share_target}" if base_url else None,
+                    'gateway_view': f"{gateway_base}/ipfs/{content.encrypted_cid}" if gateway_base else None,
+                },
+            })
+        if len(distribution_nodes) > 1:
+            distribution_nodes.sort(key=lambda entry: (0 if entry.get('is_local') else 1, entry.get('host') or ''))
+
         upload_state_norm = (latest_upload.state or '').lower() if latest_upload else ''
         conversion_state_norm = (conversion_state or '').lower() if conversion_state else ''
         ipfs_state_norm = (ipfs_sync.pin_state or '').lower() if (ipfs_sync and ipfs_sync.pin_state) else ''
@@ -656,6 +784,7 @@ async def s_api_v1_admin_uploads(request):
         }
 
         search_parts: List[Any] = [
+            content.artist,
             content.title,
             content.description,
             content.encrypted_cid,
@@ -697,6 +826,7 @@ async def s_api_v1_admin_uploads(request):
             'metadata_cid': metadata_cid,
             'content_hash': content_hash,
             'title': content.title,
+            'artist': content.artist,
             'description': content.description,
             'content_type': content.content_type,
             'size': {
@@ -733,6 +863,10 @@ async def s_api_v1_admin_uploads(request):
                 'download_primary': primary_download,
                 'download_derivatives': derivative_downloads,
             },
+            'distribution': {
+                'local_present': bool(stored),
+                'nodes': distribution_nodes,
+            },
             'flags': flags,
         })
 
@@ -821,6 +955,7 @@ async def s_api_v1_admin_users(request):
             'items': [],
             'summary': {
                 'users_returned': 0,
+                'admins_total': 0,
                 'wallets_total': 0,
                 'wallets_active': 0,
                 'licenses_total': 0,
@@ -828,6 +963,7 @@ async def s_api_v1_admin_users(request):
                 'stars_total': 0,
                 'stars_paid': 0,
                 'stars_unpaid': 0,
+                'stars_amount_total': 0,
                 'stars_amount_paid': 0,
                 'stars_amount_unpaid': 0,
                 'unique_ips_total': 0,
@@ -994,6 +1130,7 @@ async def s_api_v1_admin_users(request):
     items: List[Dict[str, Any]] = []
     summary = {
         'users_returned': 0,
+        'admins_total': 0,
         'wallets_total': 0,
         'wallets_active': 0,
         'licenses_total': 0,
@@ -1009,6 +1146,8 @@ async def s_api_v1_admin_users(request):
 
     for user in user_rows:
         summary['users_returned'] += 1
+        if getattr(user, 'is_admin', False):
+            summary['admins_total'] += 1
         meta = user.meta or {}
 
         wallet_list = wallet_map.get(user.id, [])
@@ -1077,6 +1216,7 @@ async def s_api_v1_admin_users(request):
             'created_at': _format_dt(user.created),
             'updated_at': _format_dt(user.updated),
             'last_use': _format_dt(user.last_use),
+            'is_admin': bool(user.is_admin),
             'meta': {
                 'ref_id': meta.get('ref_id'),
                 'referrer_id': meta.get('referrer_id'),
@@ -1111,6 +1251,59 @@ async def s_api_v1_admin_users(request):
     return response.json(base_payload)
 
 
+async def s_api_v1_admin_users_setadmin(request):
+    if (unauth := _ensure_admin(request)):
+        return unauth
+
+    data = request.json or {}
+    try:
+        user_id = int(data.get('user_id'))
+    except (TypeError, ValueError):
+        return response.json({"error": "BAD_USER_ID"}, status=400)
+
+    is_admin_raw = data.get('is_admin')
+    if isinstance(is_admin_raw, str):
+        normalized = is_admin_raw.strip().lower()
+        if normalized in {'1', 'true', 'yes', 'y', 'on'}:
+            is_admin = True
+        elif normalized in {'0', 'false', 'no', 'n', 'off'}:
+            is_admin = False
+        else:
+            return response.json({"error": "BAD_FLAG"}, status=400)
+    else:
+        is_admin = bool(is_admin_raw)
+
+    session = request.ctx.db_session
+    user = await session.get(User, user_id)
+    if not user:
+        return response.json({"error": "NOT_FOUND"}, status=404)
+
+    user.is_admin = is_admin
+    user.updated = datetime.utcnow()
+    try:
+        await record_event(
+            session,
+            'user_role_changed',
+            {
+                'user_id': user.id,
+                'telegram_id': user.telegram_id,
+                'is_admin': is_admin,
+            },
+            origin_host=PROJECT_HOST,
+        )
+    except Exception as exc:
+        make_log('Admin', f"Failed to record user_role_changed event: {exc}", level='warning')
+
+    await session.commit()
+    return response.json({
+        "ok": True,
+        "user": {
+            "id": user.id,
+            "is_admin": bool(user.is_admin),
+        }
+    })
+
+
 async def s_api_v1_admin_licenses(request):
     if (unauth := _ensure_admin(request)):
         return unauth
@@ -1153,20 +1346,24 @@ async def s_api_v1_admin_licenses(request):
             filters.append(UserContent.status.in_(status_values))
             applied_filters['status'] = status_values
 
+    license_type_field = cast(UserContent.meta['license_type'], String)
+
     if license_type_param:
-        lt_values: List[int] = []
+        lt_values: List[str] = []
         for part in license_type_param.split(','):
-            part = part.strip()
-            if not part:
-                continue
-            try:
-                lt_values.append(int(part))
-            except ValueError:
-                continue
+            part = (part or '').strip()
+            if part:
+                lt_values.append(part)
         if lt_values:
-            license_type_expr = cast(UserContent.meta['license_type'].astext, Integer)
-            filters.append(license_type_expr.in_(lt_values))
-            applied_filters['license_type'] = lt_values
+            clauses = []
+            plain_values = [value for value in lt_values if value.lower() not in {'unknown', 'null'}]
+            if plain_values:
+                clauses.append(license_type_field.in_(plain_values))
+            if any(value.lower() in {'unknown', 'null'} for value in lt_values):
+                clauses.append(license_type_field.is_(None))
+            if clauses:
+                filters.append(or_(*clauses))
+                applied_filters['license_type'] = lt_values
 
     if user_id_param:
         try:
@@ -1291,14 +1488,14 @@ async def s_api_v1_admin_licenses(request):
     type_counts_rows = (await session.execute(type_stmt)).all()
     type_counts = {ctype or 'unknown': int(count or 0) for ctype, count in type_counts_rows}
 
-    license_type_expr = func.coalesce(cast(UserContent.meta['license_type'].astext, Integer), -1)
-    license_type_stmt = select(license_type_expr.label('license_type'), func.count()).group_by('license_type')
+    license_type_expr = func.coalesce(license_type_field, 'unknown')
+    license_type_stmt = select(license_type_expr.label('license_type'), func.count()).group_by(license_type_expr)
     if filters:
         license_type_stmt = license_type_stmt.where(and_(*filters))
     license_type_counts_rows = (await session.execute(license_type_stmt)).all()
     license_type_counts: Dict[str, int] = {}
     for lt_value, count in license_type_counts_rows:
-        key = 'unknown' if lt_value in (None, -1) else str(int(lt_value))
+        key = 'unknown' if lt_value in (None, 'null', 'None') else str(lt_value)
         license_type_counts[key] = int(count or 0)
 
     items: List[Dict[str, Any]] = []
@@ -1338,10 +1535,21 @@ async def s_api_v1_admin_licenses(request):
             metadata_candidate = stored_meta.get('metadata') if isinstance(stored_meta.get('metadata'), dict) else {}
             title_candidates = [
                 stored_meta.get('title'),
+                metadata_candidate.get('title') if isinstance(metadata_candidate, dict) else None,
                 metadata_candidate.get('name') if isinstance(metadata_candidate, dict) else None,
                 stored_meta.get('license', {}).get('title') if isinstance(stored_meta.get('license'), dict) else None,
             ]
             title_value = next((value for value in title_candidates if isinstance(value, str) and value.strip()), None)
+            artist_candidates = []
+            if isinstance(metadata_candidate, dict):
+                artist_candidates.extend([
+                    metadata_candidate.get('artist'),
+                    (metadata_candidate.get('authors') or [None])[0] if isinstance(metadata_candidate.get('authors'), list) else None,
+                ])
+            artist_candidates.extend([
+                stored_meta.get('artist'),
+            ])
+            artist_value = next((value for value in artist_candidates if isinstance(value, str) and value.strip()), None)
             try:
                 cid_value = stored_content.cid.serialize_v2()
             except Exception:
@@ -1350,7 +1558,8 @@ async def s_api_v1_admin_licenses(request):
                 'id': stored_content.id,
                 'hash': stored_content.hash,
                 'cid': cid_value,
-                'title': title_value or stored_content.hash,
+                'title': (title_value or stored_content.hash),
+                'artist': artist_value,
                 'type': stored_content.type,
                 'owner_address': stored_content.owner_address,
                 'onchain_index': stored_content.onchain_index,
@@ -1593,6 +1802,15 @@ async def s_api_v1_admin_stars(request):
             'amount': invoice.amount,
             'paid': bool(invoice.paid),
             'invoice_url': invoice.invoice_url,
+            'telegram_id': invoice.telegram_id,
+            'bot_username': invoice.bot_username,
+            'payment_node': {
+                'public_key': invoice.payment_node_id,
+                'host': invoice.payment_node_public_host,
+            },
+            'payment_tx_id': invoice.payment_tx_id,
+            'paid_at': _format_dt(invoice.paid_at),
+            'is_remote': bool(invoice.is_remote),
             'created_at': _format_dt(invoice.created),
             'user': user_payload,
             'content': content_payload,
@@ -1614,6 +1832,136 @@ async def s_api_v1_admin_stars(request):
     return response.json(base_payload)
 
 
+async def s_api_v1_admin_events(request):
+    if (unauth := _ensure_admin(request)):
+        return unauth
+
+    session = request.ctx.db_session
+
+    try:
+        limit = int(request.args.get('limit') or 50)
+    except (TypeError, ValueError):
+        limit = 50
+    limit = max(1, min(limit, 200))
+
+    try:
+        offset = int(request.args.get('offset') or 0)
+    except (TypeError, ValueError):
+        offset = 0
+    offset = max(0, offset)
+
+    type_param = (request.args.get('type') or '').strip()
+    status_param = (request.args.get('status') or '').strip()
+    origin_param = (request.args.get('origin') or '').strip()
+    search_param = (request.args.get('search') or '').strip()
+
+    filters = []
+    applied_filters: Dict[str, Any] = {}
+
+    if type_param:
+        type_values = [value.strip() for value in type_param.split(',') if value.strip()]
+        if type_values:
+            filters.append(NodeEvent.event_type.in_(type_values))
+            applied_filters['type'] = type_values
+
+    if status_param:
+        status_values = [value.strip() for value in status_param.split(',') if value.strip()]
+        if status_values:
+            filters.append(NodeEvent.status.in_(status_values))
+            applied_filters['status'] = status_values
+
+    if origin_param:
+        origin_values = [value.strip() for value in origin_param.split(',') if value.strip()]
+        if origin_values:
+            filters.append(NodeEvent.origin_public_key.in_(origin_values))
+            applied_filters['origin'] = origin_values
+
+    if search_param:
+        search_like = f"%{search_param}%"
+        filters.append(or_(
+            NodeEvent.uid.ilike(search_like),
+            cast(NodeEvent.payload, Text).ilike(search_like),
+        ))
+        applied_filters['search'] = search_param
+
+    total_stmt = select(func.count()).select_from(NodeEvent)
+    if filters:
+        total_stmt = total_stmt.where(and_(*filters))
+    total = (await session.execute(total_stmt)).scalar_one()
+
+    query_stmt = (
+        select(NodeEvent)
+        .order_by(NodeEvent.created_at.desc())
+        .offset(offset)
+        .limit(limit)
+    )
+    if filters:
+        query_stmt = query_stmt.where(and_(*filters))
+
+    rows = (await session.execute(query_stmt)).scalars().all()
+
+    def _event_links(row: NodeEvent) -> Dict[str, Optional[str]]:
+        links: Dict[str, Optional[str]] = {}
+        payload = row.payload or {}
+        cid = payload.get('encrypted_cid') or payload.get('content_cid') or payload.get('content_id')
+        if cid:
+            links['admin_uploads'] = f"uploads?search={cid}"
+            if PROJECT_HOST:
+                links['content_view'] = f"{PROJECT_HOST}/viewContent?content={cid}"
+        invoice_id = payload.get('invoice_id')
+        if invoice_id:
+            links['admin_stars'] = f"stars?search={invoice_id}"
+        user_id = payload.get('user_id')
+        telegram_id = payload.get('telegram_id')
+        if user_id:
+            links['admin_user'] = f"users?search={user_id}"
+        elif telegram_id:
+            links['admin_user'] = f"users?search={telegram_id}"
+        return links
+
+    items: List[Dict[str, Any]] = []
+    for row in rows:
+        items.append({
+            'id': row.id,
+            'origin_public_key': row.origin_public_key,
+            'origin_host': row.origin_host,
+            'seq': int(row.seq),
+            'uid': row.uid,
+            'event_type': row.event_type,
+            'status': row.status,
+            'created_at': _format_dt(row.created_at),
+            'received_at': _format_dt(row.received_at),
+            'applied_at': _format_dt(row.applied_at),
+            'payload': row.payload or {},
+            'links': _event_links(row),
+        })
+
+    type_stmt = select(NodeEvent.event_type, func.count()).group_by(NodeEvent.event_type)
+    status_stmt = select(NodeEvent.status, func.count()).group_by(NodeEvent.status)
+    origin_stmt = select(NodeEvent.origin_public_key, func.count()).group_by(NodeEvent.origin_public_key)
+    if filters:
+        type_stmt = type_stmt.where(and_(*filters))
+        status_stmt = status_stmt.where(and_(*filters))
+        origin_stmt = origin_stmt.where(and_(*filters))
+    type_rows = (await session.execute(type_stmt)).all()
+    status_rows = (await session.execute(status_stmt)).all()
+    origin_rows = (await session.execute(origin_stmt)).all()
+
+    payload = {
+        'total': int(total or 0),
+        'limit': limit,
+        'offset': offset,
+        'filters': applied_filters,
+        'items': items,
+        'available_filters': {
+            'types': {event_type or 'unknown': int(count or 0) for event_type, count in type_rows},
+            'statuses': {status or 'unknown': int(count or 0) for status, count in status_rows},
+            'origins': {origin or 'unknown': int(count or 0) for origin, count in origin_rows},
+        }
+    }
+    return response.json(payload)
+
+
 async def s_api_v1_admin_system(request):
     if (unauth := _ensure_admin(request)):
         return unauth
@@ -1639,6 +1987,19 @@ async def s_api_v1_admin_system(request):
         'LOG_LEVEL': os.getenv('LOG_LEVEL'),
         'TESTNET': os.getenv('TESTNET'),
     }
+    telegram_bots: List[Dict[str, Any]] = []
+    if TELEGRAM_BOT_USERNAME:
+        telegram_bots.append({
+            'role': 'uploader',
+            'username': TELEGRAM_BOT_USERNAME,
+            'url': f'https://t.me/{TELEGRAM_BOT_USERNAME}',
+        })
+    if CLIENT_TELEGRAM_BOT_USERNAME:
+        telegram_bots.append({
+            'role': 'client',
+            'username': CLIENT_TELEGRAM_BOT_USERNAME,
+            'url': f'https://t.me/{CLIENT_TELEGRAM_BOT_USERNAME}',
+        })
 
     blockchain_counts_rows = (await session.execute(
         select(BlockchainTask.status, func.count()).group_by(BlockchainTask.status)
@@ -1662,6 +2023,7 @@ async def s_api_v1_admin_system(request):
         'services': _service_states(request),
         'blockchain_tasks': blockchain_counts,
         'latest_index_items': index_entries,
+        'telegram_bots': telegram_bots,
     }
     return response.json(payload)
 
@@ -1721,7 +2083,7 @@ async def s_api_v1_admin_node_setrole(request):
         row = (await session.execute(select(KnownNode).where(KnownNode.ip == host))).scalars().first()
     if not row:
         return response.json({"error": "NOT_FOUND"}, status=404)
-    meta = row.meta or {}
+    meta = {**(row.meta or {})}
     meta['role'] = role
     row.meta = meta
     await session.commit()
@@ -1769,11 +2131,17 @@ async def s_api_v1_admin_status(request):
     ec = (await session.execute(select(EncryptedContent))).scalars().all()
     backlog = 0
     for e in ec:
-        if not e.preview_enabled:
+        ctype = (e.content_type or '').lower()
+        if ctype.startswith('audio/'):
+            req = {'decrypted_low', 'decrypted_high'}
+        elif ctype.startswith('video/'):
+            req = {'decrypted_low', 'decrypted_high', 'decrypted_preview'}
+        else:
+            req = {'decrypted_original'}
+        if not req:
             continue
-        kinds = [d.kind for d in deriv if d.content_id == e.id and d.status == 'ready']
-        req = {'decrypted_low', 'decrypted_high', 'decrypted_preview'}
-        if not req.issubset(set(kinds)):
+        kinds = {d.kind for d in deriv if d.content_id == e.id and d.status == 'ready'}
+        if not req.issubset(kinds):
             backlog += 1
     try:
         bs = await bitswap_stat()
@@ -1868,6 +2236,272 @@ async def s_api_v1_admin_cache_cleanup(request):
     return response.json({"ok": True, "removed": removed})
 
 
+async def s_api_v1_admin_network(request):
+    """Сводка состояния децентрализованной сети для вкладки "Состояние сети".
+
+    Возвращает:
+    - summary: n_estimate, количество участников, число островов, сводка конфликтов репликаций
+    - members: список нод с ролями/версиями/достижимостью и атрибутами
+    - per_node_replication: сколько лизов держит каждая нода и сколько раз является лидером
+    """
+    if (unauth := _ensure_admin(request)):
+        return unauth
+
+    mem = getattr(request.app.ctx, 'memory', None)
+    if not mem:
+        return response.json({"error": "MEMORY_NOT_READY"}, status=503)
+
+    membership = mem.membership.state
+    n_est = membership.n_estimate()
+    active_all = membership.active_members(include_islands=True)
+    active_filtered = membership.active_members(include_islands=False)
+    islands = [m for m in active_all if membership.reachability_ratio(m['node_id']) < dht_config.default_q]
+
+    # Обогащение из БД (версии, роли, public_host)
+    db = request.ctx.db_session
+    known = (await db.execute(select(KnownNode))).scalars().all()
+    meta_by_pub = {r.public_key: (r, r.meta or {}) for r in known}
+    meta_by_host = {r.ip: (r, r.meta or {}) for r in known}
+
+    # Precompute receipts stats per node
+    receipts_elements = membership.receipts.elements() if hasattr(membership, 'receipts') else {}
+    receipts_by_target: Dict[str, Dict[str, Any]] = {}
+    for _rid, rec in receipts_elements.items():
+        tid = str(rec.get('target_id'))
+        if not tid:
+            continue
+        bucket = receipts_by_target.setdefault(tid, { 'total': 0, 'asn_set': set() })
+        bucket['total'] += 1
+        if rec.get('asn') is not None:
+            try:
+                bucket['asn_set'].add(int(rec.get('asn')))
+            except Exception:
+                pass
+
+    def _enrich(member: dict) -> dict:
+        pub = str(member.get('public_key') or '')
+        host = str(member.get('ip') or '')
+        row_meta = (meta_by_pub.get(pub) or meta_by_host.get(host) or (None, {}))[1]
+        caps = (member.get('meta') or {}).get('capabilities') or {}
+        rec_stat = receipts_by_target.get(member.get('node_id') or '', {'total': 0, 'asn_set': set()})
+        return {
+            'node_id': member.get('node_id'),
+            'public_key': pub or None,
+            'public_host': row_meta.get('public_host'),
+            'version': row_meta.get('version'),
+            'role': row_meta.get('role') or 'read-only',
+            'ip': host or None,
+            'asn': member.get('asn'),
+            'ip_first_octet': member.get('ip_first_octet'),
+            'reachability_ratio': membership.reachability_ratio(member.get('node_id')), 
+            'last_update': member.get('last_update'),
+            'accepts_inbound': bool(caps.get('accepts_inbound')),
+            'is_bootstrap': bool(caps.get('is_bootstrap')),
+            'receipts_total': int(rec_stat.get('total') or 0),
+            'receipts_asn_unique': len(rec_stat.get('asn_set') or ()),
+        }
+
+    members_payload = [_enrich(m) for m in active_all]
+    # Server-side pagination
+    try:
+        page = max(1, int(request.args.get('page') or 1))
+    except Exception:
+        page = 1
+    try:
+        page_size = max(1, min(500, int(request.args.get('page_size') or 100)))
+    except Exception:
+        page_size = 100
+    total_members = len(members_payload)
+    start = (page - 1) * page_size
+    end = start + page_size
+    members_page = members_payload[start:end]
+
+    # Агрегация репликаций по снимку DHT
+    snapshot = mem.dht_store.snapshot() if hasattr(mem, 'dht_store') else {}
+    per_node = {}
+    conflict_under = 0
+    conflict_over = 0
+    for fp, rec in snapshot.items():
+        key = rec.get('key') or ''
+        if not key.startswith('meta:'):
+            continue
+        value = rec.get('value') or {}
+        content_id = value.get('content_id')
+        leases = (value.get('replica_leases') or {}).values()
+        leader = value.get('leader')
+        # Конфликты
+        for ev in value.get('conflict_log') or []:
+            t = (ev.get('type') or '').upper()
+            if t == 'UNDER_REPLICATED':
+                conflict_under += 1
+            elif t == 'OVER_REPLICATED':
+                conflict_over += 1
+            # Пер-нодовые конфликты
+            nid = ev.get('node_id')
+            if nid:
+                p = per_node.setdefault(nid, {'leases_held': 0, 'leaderships': 0, 'sample_contents': [], 'conflicts': {'over': 0, 'lease_expired': 0}, 'conflict_samples': []})
+                if t == 'OVER_REPLICATED':
+                    p['conflicts']['over'] = p['conflicts'].get('over', 0) + 1
+                elif t == 'LEASE_EXPIRED':
+                    p['conflicts']['lease_expired'] = p['conflicts'].get('lease_expired', 0) + 1
+                if content_id and len(p['conflict_samples']) < 10:
+                    p['conflict_samples'].append({'content_id': content_id, 'type': t, 'ts': ev.get('ts')})
+        # Лизы
+        for l in leases:
+            nid = l.get('node_id')
+            if not nid:
+                continue
+            p = per_node.setdefault(nid, {'leases_held': 0, 'leaderships': 0, 'sample_contents': [], 'conflicts': {'over': 0, 'lease_expired': 0}, 'conflict_samples': []})
+            p['leases_held'] += 1
+            if content_id and len(p['sample_contents']) < 5:
+                p['sample_contents'].append(content_id)
+        if leader:
+            p = per_node.setdefault(leader, {'leases_held': 0, 'leaderships': 0, 'sample_contents': [], 'conflicts': {'over': 0, 'lease_expired': 0}, 'conflict_samples': []})
+            p['leaderships'] += 1
+
+    # Добавим trusted-only n_estimate и показатели активности в summary
+    # Соберём allowed_nodes так же, как в репликации
+    from app.core._utils.b58 import b58decode
+    from app.core.network.dht.crypto import compute_node_id
+    allowed_nodes = set()
+    for row, meta in meta_by_pub.values():
+        try:
+            if (meta or {}).get('role') == 'trusted' and row.public_key:
+                allowed_nodes.add(compute_node_id(b58decode(row.public_key)))
+        except Exception:
+            pass
+    allowed_nodes.add(mem.node_id)
+    n_est_trusted = membership.n_estimate_trusted(allowed_nodes) if hasattr(membership, 'n_estimate_trusted') else n_est
+    # Активные trusted: те, кто в allowed_nodes и проходят TTL/Q
+    active_trusted = [m for m in active_filtered if m.get('node_id') in allowed_nodes]
+    # Экспортируем конфиг интервалов
+    from app.core.network.dht import dht_config
+
+    # Build receipts report with validation status
+    receipts_raw = (membership.receipts.elements() if hasattr(membership, 'receipts') else {}) or {}
+    receipts: List[Dict[str, Any]] = []
+    members_map = membership.members.elements() if hasattr(membership, 'members') else {}
+    for _rid, entry in receipts_raw.items():
+        target_id = str(entry.get('target_id'))
+        issuer_id = str(entry.get('issuer_id'))
+        asn = entry.get('asn')
+        timestamp = entry.get('timestamp')
+        signature = str(entry.get('signature') or '')
+        status = 'unknown'
+        # verify if possible
+        issuer_pub = None
+        for mid, mdata in members_map.items():
+            if mid == issuer_id:
+                issuer_pub = mdata.get('public_key')
+                break
+        if issuer_pub:
+            try:
+                from app.core._utils.b58 import b58decode as _b58d
+                from app.core.network.dht.crypto import compute_node_id
+                import nacl.signing  # type: ignore
+                # node_id/pubkey match
+                if compute_node_id(_b58d(issuer_pub)) != issuer_id:
+                    status = 'mismatch_node_id'
+                else:
+                    payload = {
+                        'schema_version': dht_config.schema_version,
+                        'target_id': target_id,
+                        'issuer_id': issuer_id,
+                        'asn': int(asn) if asn is not None else None,
+                        'timestamp': float(timestamp or 0),
+                    }
+                    blob = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()
+                    vk = nacl.signing.VerifyKey(_b58d(issuer_pub))
+                    vk.verify(blob, _b58d(signature))
+                    status = 'valid'
+            except Exception:
+                status = 'bad_signature'
+        else:
+            status = 'unknown_issuer'
+        receipts.append({
+            'target_id': target_id,
+            'issuer_id': issuer_id,
+            'asn': asn,
+            'timestamp': timestamp,
+            'status': status,
+        })
+
+    return response.json({
+        'summary': {
+            'n_estimate': n_est,
+            'n_estimate_trusted': n_est_trusted,
+            'active_trusted': len(active_trusted),
+            'members_total': len(active_all),
+            'active': len(active_filtered),
+            'islands': len(islands),
+            'replication_conflicts': {
+                'under': conflict_under,
+                'over': conflict_over,
+            },
+            'config': {
+                'heartbeat_interval': dht_config.heartbeat_interval,
+                'lease_ttl': dht_config.lease_ttl,
+                'gossip_interval_sec': dht_config.gossip_interval_sec,
+                'gossip_backoff_base_sec': dht_config.gossip_backoff_base_sec,
+                'gossip_backoff_cap_sec': dht_config.gossip_backoff_cap_sec,
+            }
+        },
+        'members': members_page,
+        'per_node_replication': per_node,
+        'receipts': receipts,
+        'paging': { 'page': page, 'page_size': page_size, 'total': total_members },
+    })
+
+
+async def s_api_v1_admin_network_config(request):
+    if (unauth := _ensure_admin(request)):
+        return unauth
+    cfg = dht_config
+    async with request.ctx.db_session() as session:
+        sc = ServiceConfig(session)
+        out = {
+            'heartbeat_interval': cfg.heartbeat_interval,
+            'lease_ttl': cfg.lease_ttl,
+            'gossip_interval_sec': cfg.gossip_interval_sec,
+            'gossip_backoff_base_sec': cfg.gossip_backoff_base_sec,
+            'gossip_backoff_cap_sec': cfg.gossip_backoff_cap_sec,
+        }
+        # include overrides if present
+        for k in list(out.keys()):
+            ov = await sc.get(f'DHT_{k.upper()}', None)
+            if ov is not None:
+                out[k] = int(ov)
+    return response.json({'ok': True, 'config': out})
+
+
+async def s_api_v1_admin_network_config_set(request):
+    if (unauth := _ensure_admin(request)):
+        return unauth
+    data = request.json or {}
+    allowed = {
+        'heartbeat_interval': (5, 3600),
+        'lease_ttl': (60, 86400),
+        'gossip_interval_sec': (5, 600),
+        'gossip_backoff_base_sec': (1, 300),
+        'gossip_backoff_cap_sec': (10, 7200),
+    }
+    updates = {}
+    for key, (lo, hi) in allowed.items():
+        if key in data:
+            try:
+                val = int(data[key])
+            except Exception:
+                return response.json({'error': f'BAD_{key.upper()}'}, status=400)
+            if val < lo or val > hi:
+                return response.json({'error': f'RANGE_{key.upper()}', 'min': lo, 'max': hi}, status=400)
+            updates[key] = val
+    async with request.ctx.db_session() as session:
+        sc = ServiceConfig(session)
+        for key, val in updates.items():
+            await sc.set(f'DHT_{key.upper()}', val)
+    return response.json({'ok': True, 'updated': updates})
+
+
 async def s_api_v1_admin_sync_setlimits(request):
     if (unauth := _ensure_admin(request)):
         return unauth

app/api/routes/auth.py

@@ -106,12 +106,17 @@ async def s_api_v1_auth_twa(request):
                 user_id=known_user.id,
                 network='ton',
                 wallet_key='web2-client==1',
-                connection_id=connection_payload,
+                # `ton_proof.payload` is expected to be single-use in many wallets (and it is unique per auth call here),
+                # but client-side retries/replays can happen; keep payload separately and make DB id unique.
+                connection_id=f"{connection_payload}.{uuid4().hex}",
                 wallet_address=Address(wallet_info.account.address).to_string(1, 1, 1),
                 keys={
-                    'ton_proof': auth_data['ton_proof']
+                    'ton_proof': auth_data['ton_proof'],
+                    'ton_proof_payload': connection_payload,
+                },
+                meta={
+                    'ton_proof_payload': connection_payload,
                 },
-                meta={},
                 created=datetime.now(),
                 updated=datetime.now(),
                 invalidated=False,

app/api/routes/content.py

@@ -1,6 +1,7 @@
+from __future__ import annotations
 from datetime import datetime, timedelta
 from sanic import response
-from sqlalchemy import select, and_, func
+from sqlalchemy import select, and_, func, or_
 from aiogram import Bot, types
 from sqlalchemy import and_
 from app.core.logger import make_log
@@ -9,10 +10,13 @@ from app.core.models.node_storage import StoredContent
 from app.core.models.keys import KnownKey
 from app.core.models import StarsInvoice
 from app.core.models.content.user_content import UserContent
-from app.core._config import CLIENT_TELEGRAM_API_KEY, PROJECT_HOST
+from app.core._config import CLIENT_TELEGRAM_API_KEY, CLIENT_TELEGRAM_BOT_USERNAME, PROJECT_HOST
 from app.core.models.content_v3 import EncryptedContent as ECv3, ContentDerivative as CDv3, UploadSession
 from app.core.content.content_id import ContentId
+from app.core.network.dht import MetricsAggregator
+import os
 import json
+import time
 import uuid
 
 
@@ -79,12 +83,35 @@ async def s_api_v1_content_view(request, content_address: str):
             content_type = ctype.split('/')[0]
         except Exception:
             content_type = 'application'
-        return {'encrypted_content': encrypted, 'decrypted_content': decrypted, 'content_type': content_type}
-    content = await open_content_async(request.ctx.db_session, r_content)
+        return {
+            'encrypted_content': encrypted,
+            'decrypted_content': decrypted,
+            'content_type': content_type,
+            'content_mime': ctype,
+        }
+    try:
+        content = await open_content_async(request.ctx.db_session, r_content)
+    except AssertionError:
+        # Fallback: handle plain stored content without encrypted/decrypted pairing
+        sc = r_content
+        from mimetypes import guess_type as _guess
+        _mime, _ = _guess(sc.filename or '')
+        _mime = _mime or 'application/octet-stream'
+        try:
+            _ctype = _mime.split('/')[0]
+        except Exception:
+            _ctype = 'application'
+        content = {
+            'encrypted_content': sc,
+            'decrypted_content': sc,
+            'content_type': _ctype,
+            'content_mime': _mime,
+        }
 
     master_address = content['encrypted_content'].meta.get('item_address', '')
     opts = {
         'content_type': content['content_type'],  # возможно с ошибками, нужно переделать на ffprobe
+        'content_mime': content.get('content_mime'),
         'content_address': license_address or master_address,
         'license_address': license_address,
         'master_address': master_address,
@@ -103,18 +130,26 @@ async def s_api_v1_content_view(request, content_address: str):
     have_access = False
     if request.ctx.user:
         user_wallet_address = await request.ctx.user.wallet_address_async(request.ctx.db_session)
+        user_telegram_id = getattr(request.ctx.user, 'telegram_id', None)
+        or_clauses = [StarsInvoice.user_id == request.ctx.user.id]
+        if user_telegram_id is not None:
+            or_clauses.append(StarsInvoice.telegram_id == user_telegram_id)
+        stars_access = False
+        if or_clauses:
+            stars_access = bool((await request.ctx.db_session.execute(select(StarsInvoice).where(
+                and_(
+                    StarsInvoice.content_hash == content['encrypted_content'].hash,
+                    StarsInvoice.paid.is_(True),
+                    or_(*or_clauses)
+                )
+            ))).scalars().first())
+
         have_access = (
                 (content['encrypted_content'].owner_address == user_wallet_address)
                 or bool((await request.ctx.db_session.execute(select(UserContent).where(
                     and_(UserContent.owner_address == user_wallet_address, UserContent.status == 'active', UserContent.content_id == content['encrypted_content'].id)
-                ))).scalars().first()) \
-                or bool((await request.ctx.db_session.execute(select(StarsInvoice).where(
-                    and_(
-                        StarsInvoice.user_id == request.ctx.user.id,
-                        StarsInvoice.content_hash == content['encrypted_content'].hash,
-                        StarsInvoice.paid == True
-                    )
                 ))).scalars().first())
+                or stars_access
         )
 
         if not have_access:
@@ -123,8 +158,10 @@ async def s_api_v1_content_view(request, content_address: str):
                 current_star_rate = 0.00000001
 
             stars_cost = int(int(content['encrypted_content'].meta['license']['resale']['price']) / 1e9 / current_star_rate * 1.2)
-            if request.ctx.user.telegram_id in [5587262915, 6861699286]:
+            if getattr(request.ctx.user, 'is_admin', False):
                 stars_cost = 2
+            else:
+                stars_cost = int(int(content['encrypted_content'].meta['license']['resale']['price']) / 1e9 / current_star_rate * 1.2)
 
             invoice_id = f"access_{uuid.uuid4().hex}"
             exist_invoice = (await request.ctx.db_session.execute(select(StarsInvoice).where(
@@ -155,7 +192,9 @@ async def s_api_v1_content_view(request, content_address: str):
                             amount=stars_cost,
                             user_id=request.ctx.user.id,
                             content_hash=content['encrypted_content'].hash,
-                            invoice_url=invoice_url
+                            invoice_url=invoice_url,
+                            telegram_id=getattr(request.ctx.user, 'telegram_id', None),
+                            bot_username=CLIENT_TELEGRAM_BOT_USERNAME,
                         )
                     )
                     await request.ctx.db_session.commit()
@@ -168,12 +207,21 @@ async def s_api_v1_content_view(request, content_address: str):
                     'amount': stars_cost,
                 }
 
-    display_options = {'content_url': None}
+    display_options = {
+        'content_url': None,
+        'content_kind': None,
+        'has_preview': False,
+        'original_available': False,
+        'requires_license': False,
+    }
 
     if have_access:
         opts['have_licenses'].append('listen')
 
-    enc_cid = content['encrypted_content'].meta.get('content_cid') or content['encrypted_content'].meta.get('encrypted_cid')
+    encrypted_json = content['encrypted_content'].json_format()
+    decrypted_json = content['decrypted_content'].json_format()
+
+    enc_cid = encrypted_json.get('content_cid') or encrypted_json.get('encrypted_cid')
     ec_v3 = None
     derivative_rows = []
     if enc_cid:
@@ -187,6 +235,40 @@ async def s_api_v1_content_view(request, content_address: str):
 
     converted_meta_map = dict(content['encrypted_content'].meta.get('converted_content') or {})
 
+    content_mime = (
+        (ec_v3.content_type if ec_v3 and ec_v3.content_type else None)
+        or decrypted_json.get('content_type')
+        or encrypted_json.get('content_type')
+        or opts.get('content_mime')
+        or 'application/octet-stream'
+    )
+    # Fallback: if stored content reports generic application/*, try guess by filename
+    try:
+        if content_mime.startswith('application/'):
+            from mimetypes import guess_type as _guess
+            _fn = decrypted_json.get('filename') or encrypted_json.get('filename') or ''
+            _gm, _ = _guess(_fn)
+            if _gm:
+                content_mime = _gm
+    except Exception:
+        pass
+    opts['content_mime'] = content_mime
+    try:
+        opts['content_type'] = content_mime.split('/')[0]
+    except Exception:
+        opts['content_type'] = opts.get('content_type') or 'application'
+
+    content_kind = 'audio'
+    if content_mime.startswith('video/'):
+        content_kind = 'video'
+    elif content_mime.startswith('audio/'):
+        content_kind = 'audio'
+    else:
+        content_kind = 'binary'
+
+    display_options['content_kind'] = content_kind
+    display_options['requires_license'] = (not have_access) and content_kind == 'binary'
+
     derivative_latest = {}
     if derivative_rows:
         derivative_sorted = sorted(derivative_rows, key=lambda row: row.created_at or datetime.min)
@@ -197,10 +279,17 @@ async def s_api_v1_content_view(request, content_address: str):
         if not row or not row.local_path:
             return None, None
         file_hash = row.local_path.split('/')[-1]
-        return file_hash, f"{PROJECT_HOST}/api/v1.5/storage/{file_hash}"
+        return file_hash, f"{PROJECT_HOST}/api/v1/storage.proxy/{file_hash}"
+
+    has_preview = bool(derivative_latest.get('decrypted_preview') or converted_meta_map.get('low_preview'))
+    display_options['has_preview'] = has_preview
+    display_options['original_available'] = bool(derivative_latest.get('decrypted_original') or converted_meta_map.get('original'))
 
     chosen_row = None
-    if have_access:
+    if content_kind == 'binary':
+        if have_access and 'decrypted_original' in derivative_latest:
+            chosen_row = derivative_latest['decrypted_original']
+    elif have_access:
         for key in ('decrypted_low', 'decrypted_high'):
             if key in derivative_latest:
                 chosen_row = derivative_latest[key]
@@ -211,27 +300,93 @@ async def s_api_v1_content_view(request, content_address: str):
                 chosen_row = derivative_latest[key]
                 break
 
+    def _make_token_for(hash_value: str, scope: str, user_id: int | None) -> str:
+        try:
+            from app.core._crypto.signer import Signer
+            from app.core._secrets import hot_seed, hot_pubkey
+            from app.core._utils.b58 import b58encode as _b58e
+            signer = Signer(hot_seed)
+            # Media URLs are polled very frequently by the web client (e.g. every 5s).
+            # If we generate a new exp for every request, the signed URL changes every poll,
+            # forcing the player to reload and breaking continuous streaming.
+            #
+            # To keep URLs stable while still expiring tokens, we "bucket" exp time.
+            # Default behavior keeps tokens stable for ~10 minutes; can be tuned via env.
+            ttl_sec = int(os.getenv("STORAGE_PROXY_TOKEN_TTL_SEC", "600"))
+            bucket_sec = int(os.getenv("STORAGE_PROXY_TOKEN_BUCKET_SEC", str(ttl_sec)))
+            ttl_sec = max(1, ttl_sec)
+            bucket_sec = max(1, bucket_sec)
+            now = int(time.time())
+            exp_base = now + ttl_sec
+            # Always move to the next bucket boundary so the token doesn't flip immediately
+            # after a boundary due to rounding edge cases.
+            exp = ((exp_base // bucket_sec) + 1) * bucket_sec
+            uid = int(user_id or 0)
+            payload = {'hash': hash_value, 'scope': scope, 'exp': exp, 'uid': uid}
+            blob = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()
+            sig = signer.sign(blob)
+            pub = _b58e(hot_pubkey).decode()
+            return f"pub={pub}&exp={exp}&scope={scope}&uid={uid}&sig={sig}"
+        except Exception:
+            return ""
+
     if chosen_row:
         file_hash, url = _row_to_hash_and_url(chosen_row)
         if url:
-            display_options['content_url'] = url
-            opts['content_ext'] = (chosen_row.content_type or '').split('/')[-1] if chosen_row.content_type else None
-            converted_meta_map.setdefault('low' if have_access else 'low_preview', file_hash)
+            token = _make_token_for(file_hash or '', 'full' if have_access else 'preview', getattr(request.ctx.user, 'id', None))
+            display_options['content_url'] = f"{url}?{token}" if token else url
+            ext_candidate = None
+            if chosen_row.content_type:
+                ext_candidate = chosen_row.content_type.split('/')[-1]
+            elif '/' in content_mime:
+                ext_candidate = content_mime.split('/')[-1]
+            if ext_candidate:
+                opts['content_ext'] = ext_candidate
+            if content_kind == 'binary':
+                display_options['original_available'] = True
+                converted_meta_map.setdefault('original', file_hash)
+            elif have_access:
+                converted_meta_map.setdefault('low', file_hash)
+            else:
+                converted_meta_map.setdefault('low_preview', file_hash)
 
     if not display_options['content_url'] and converted_meta_map:
-        preference = ['low', 'high', 'low_preview'] if have_access else ['low_preview', 'low', 'high']
+        if content_kind == 'binary':
+            preference = ['original'] if have_access else []
+        else:
+            preference = ['low', 'high', 'low_preview'] if have_access else ['low_preview', 'low', 'high']
         for key in preference:
             hash_value = converted_meta_map.get(key)
             if not hash_value:
                 continue
-            stored = (await request.ctx.db_session.execute(select(StoredContent).where(StoredContent.hash == hash_value))).scalars().first()
-            if stored:
-                display_options['content_url'] = stored.web_url
-                opts['content_ext'] = stored.filename.split('.')[-1]
-                break
+            # Пробуем сразу через прокси (даже если локальной записи нет)
+            token = _make_token_for(hash_value, 'full' if have_access else 'preview', getattr(request.ctx.user, 'id', None))
+            display_options['content_url'] = f"{PROJECT_HOST}/api/v1/storage.proxy/{hash_value}?{token}" if token else f"{PROJECT_HOST}/api/v1/storage.proxy/{hash_value}"
+            if '/' in content_mime:
+                opts['content_ext'] = content_mime.split('/')[-1]
+            if content_kind == 'binary':
+                display_options['original_available'] = True
+            break
+
+    # Final fallback: no derivatives known — serve stored content directly for AV
+    if not display_options['content_url'] and content_kind in ('audio', 'video'):
+        from app.core._utils.b58 import b58encode as _b58e
+        scid = decrypted_json.get('cid') or encrypted_json.get('cid')
+        try:
+            from app.core.content.content_id import ContentId as _CID
+            if scid:
+                _cid = _CID.deserialize(scid)
+                h = _cid.content_hash_b58
+            else:
+                h = decrypted_json.get('hash')
+        except Exception:
+            h = decrypted_json.get('hash')
+        if h:
+            token = _make_token_for(h, 'preview' if not have_access else 'full', getattr(request.ctx.user, 'id', None))
+            display_options['content_url'] = f"{PROJECT_HOST}/api/v1/storage.proxy/{h}?{token}" if token else f"{PROJECT_HOST}/api/v1/storage.proxy/{h}"
 
     # Metadata fallback
-    content_meta = content['encrypted_content'].json_format()
+    content_meta = encrypted_json
     content_metadata_json = None
     _mcid = content_meta.get('metadata_cid') or None
     if _mcid:
@@ -254,7 +409,8 @@ async def s_api_v1_content_view(request, content_address: str):
         }
         cover_cid = content_meta.get('cover_cid')
         if cover_cid:
-            content_metadata_json.setdefault('image', f"{PROJECT_HOST}/api/v1.5/storage/{cover_cid}")
+            token = _make_token_for(cover_cid, 'preview', getattr(request.ctx.user, 'id', None))
+            content_metadata_json.setdefault('image', f"{PROJECT_HOST}/api/v1/storage.proxy/{cover_cid}?{token}" if token else f"{PROJECT_HOST}/api/v1/storage.proxy/{cover_cid}")
 
     display_options['metadata'] = content_metadata_json
 
@@ -278,9 +434,14 @@ async def s_api_v1_content_view(request, content_address: str):
             'updated_at': (row.last_access_at or row.created_at).isoformat() + 'Z' if (row.last_access_at or row.created_at) else None,
         })
 
-    required_kinds = {'decrypted_low', 'decrypted_high'}
-    if ec_v3 and ec_v3.content_type.startswith('video/'):
-        required_kinds.add('decrypted_preview')
+    required_kinds = set()
+    if content_kind == 'binary':
+        if derivative_latest.get('decrypted_original') or converted_meta_map.get('original'):
+            required_kinds.add('decrypted_original')
+    else:
+        required_kinds = {'decrypted_low', 'decrypted_high'}
+        if ec_v3 and ec_v3.content_type and ec_v3.content_type.startswith('video/'):
+            required_kinds.add('decrypted_preview')
 
     statuses_by_kind = {kind: row.status for kind, row in derivative_summary_map.items() if kind in required_kinds}
     conversion_state = 'pending'
@@ -306,15 +467,15 @@ async def s_api_v1_content_view(request, content_address: str):
             'updated_at': upload_row.updated_at.isoformat() + 'Z' if upload_row.updated_at else None,
         }
 
-    final_state = 'ready' if display_options['content_url'] else None
-    if final_state != 'ready':
-        upload_state = upload_row.state if upload_row else None
-        if conversion_state == 'failed' or upload_state in ('failed', 'conversion_failed'):
-            final_state = 'failed'
-        elif conversion_state in ('processing', 'partial') or upload_state in ('processing', 'pinned'):
-            final_state = 'processing'
-        else:
-            final_state = 'uploaded'
+    upload_state = upload_row.state if upload_row else None
+    if conversion_state == 'failed' or upload_state in ('failed', 'conversion_failed'):
+        final_state = 'failed'
+    elif conversion_state == 'ready':
+        final_state = 'ready'
+    elif conversion_state in ('processing', 'partial') or upload_state in ('processing', 'pinned'):
+        final_state = 'processing'
+    else:
+        final_state = 'uploaded'
 
     conversion_info = {
         'state': conversion_state,
@@ -329,7 +490,39 @@ async def s_api_v1_content_view(request, content_address: str):
         'state': final_state,
         'conversion_state': conversion_state,
         'upload_state': upload_info['state'] if upload_info else None,
+        'has_access': have_access,
     }
+    if not opts.get('content_ext') and '/' in content_mime:
+        opts['content_ext'] = content_mime.split('/')[-1]
+
+    metrics_mgr: MetricsAggregator | None = getattr(request.app.ctx.memory, "metrics", None)
+    if metrics_mgr:
+        viewer_salt_raw = request.headers.get("X-View-Salt")
+        if viewer_salt_raw:
+            try:
+                viewer_salt = bytes.fromhex(viewer_salt_raw)
+            except ValueError:
+                viewer_salt = viewer_salt_raw.encode()
+        elif request.ctx.user:
+            viewer_salt = f"user:{request.ctx.user.id}".encode()
+        else:
+            viewer_salt = (request.remote_addr or request.ip or "anonymous").encode()
+        try:
+            watch_time_param = int(request.args.get("watch_time", 0))
+        except (TypeError, ValueError):
+            watch_time_param = 0
+        try:
+            bytes_out_param = int(request.args.get("bytes_out", 0))
+        except (TypeError, ValueError):
+            bytes_out_param = 0
+        completed_param = request.args.get("completed", "0") in ("1", "true", "True")
+        metrics_mgr.record_view(
+            content_id=content['encrypted_content'].hash,
+            viewer_salt=viewer_salt,
+            watch_time=watch_time_param,
+            bytes_out=bytes_out_param,
+            completed=completed_param,
+        )
 
     return response.json({
         **opts,

app/api/routes/dht.py

@@ -0,0 +1,125 @@
+from __future__ import annotations
+
+import json
+from typing import Any, Dict, List
+
+from sanic import response
+
+from app.core.logger import make_log
+from app.core._utils.b58 import b58decode
+from app.core.network.dht.records import DHTRecord
+from app.core.network.dht.store import DHTStore
+from app.core.network.dht.crypto import compute_node_id
+from app.core.network.dht.keys import MetaKey, MembershipKey, MetricKey
+from sqlalchemy import select
+from app.core.models.my_network import KnownNode
+
+
+def _merge_strategy_for(key: str):
+    # Выбираем правильную стратегию merge по префиксу ключа
+    from app.core.network.dht.replication import ReplicationState
+    from app.core.network.dht.membership import MembershipState
+    from app.core.network.dht.metrics import ContentMetricsState
+    if key.startswith('meta:'):
+        return lambda a, b: ReplicationState.from_dict(a).merge_with(ReplicationState.from_dict(b)).to_dict()
+    if key.startswith('membership:'):
+        # Для membership нужен node_id, но это только для локального состояния; здесь достаточно CRDT-мерджа
+        return lambda a, b: MembershipState.from_dict('remote', None, a).merge(MembershipState.from_dict('remote', None, b)).to_dict()
+    if key.startswith('metric:'):
+        return lambda a, b: ContentMetricsState.from_dict('remote', a).merge(ContentMetricsState.from_dict('remote', b)).to_dict()
+    return lambda a, b: b
+
+
+async def s_api_v1_dht_get(request):
+    """Возвращает запись DHT по fingerprint или key."""
+    store: DHTStore = request.app.ctx.memory.dht_store
+    fp = request.args.get('fingerprint')
+    key = request.args.get('key')
+    if fp:
+        rec = store.get(fp)
+        if not rec:
+            return response.json({'error': 'NOT_FOUND'}, status=404)
+        return response.json({**rec.to_payload(), 'signature': rec.signature})
+    if key:
+        snap = store.snapshot()
+        for _fp, payload in snap.items():
+            if payload.get('key') == key:
+                return response.json(payload)
+        return response.json({'error': 'NOT_FOUND'}, status=404)
+    return response.json({'error': 'BAD_REQUEST'}, status=400)
+
+
+def _verify_publisher(node_id: str, public_key_b58: str) -> bool:
+    try:
+        derived = compute_node_id(b58decode(public_key_b58))
+        return derived == node_id
+    except Exception:
+        return False
+
+
+async def s_api_v1_dht_put(request):
+    """Принимает запись(и) DHT, проверяет подпись и выполняет merge/persist.
+
+    Поддерживает одиночную запись (record: {...}) и пакет (records: [{...}]).
+    Требует поле public_key отправителя и соответствие node_id.
+    """
+    mem = request.app.ctx.memory
+    store: DHTStore = mem.dht_store
+    data = request.json or {}
+    public_key = data.get('public_key')
+    if not public_key:
+        return response.json({'error': 'MISSING_PUBLIC_KEY'}, status=400)
+
+    # Determine publisher role (trusted/read-only/deny)
+    role = None
+    try:
+        session = request.ctx.db_session
+        kn = (await session.execute(select(KnownNode).where(KnownNode.public_key == public_key))).scalars().first()
+        role = (kn.meta or {}).get('role') if kn and kn.meta else None
+    except Exception:
+        role = None
+
+    def _process_one(payload: Dict[str, Any]) -> Dict[str, Any]:
+        try:
+            rec = DHTRecord.create(
+                key=payload['key'],
+                fingerprint=payload['fingerprint'],
+                value=payload['value'],
+                node_id=payload['node_id'],
+                logical_counter=int(payload['logical_counter']),
+                signature=payload.get('signature'),
+                timestamp=float(payload.get('timestamp') or 0),
+            )
+        except Exception as e:
+            return {'error': f'BAD_RECORD: {e}'}
+        if not _verify_publisher(rec.node_id, public_key):
+            return {'error': 'NODE_ID_MISMATCH'}
+        # Подтверждение подписи записи
+        if not rec.verify(public_key):
+            return {'error': 'BAD_SIGNATURE'}
+        # Enforce ACL: untrusted nodes may not mutate meta/metric records
+        if role != 'trusted':
+            if rec.key.startswith('meta:') or rec.key.startswith('metric:'):
+                return {'error': 'FORBIDDEN_NOT_TRUSTED'}
+        merge_fn = _merge_strategy_for(rec.key)
+        try:
+            merged = store.merge_record(rec, merge_fn)
+            return {'ok': True, 'fingerprint': merged.fingerprint}
+        except Exception as e:
+            make_log('DHT.put', f'merge failed: {e}', level='warning')
+            return {'error': 'MERGE_FAILED'}
+
+    if 'record' in data:
+        result = _process_one(data['record'])
+        status = 200 if 'ok' in result else 400
+        return response.json(result, status=status)
+    elif 'records' in data and isinstance(data['records'], list):
+        results: List[Dict[str, Any]] = []
+        ok = True
+        for item in data['records']:
+            res = _process_one(item)
+            if 'error' in res:
+                ok = False
+            results.append(res)
+        return response.json({'ok': ok, 'results': results}, status=200 if ok else 207)
+    return response.json({'error': 'BAD_REQUEST'}, status=400)

app/api/routes/metrics.py

@@ -0,0 +1,39 @@
+from __future__ import annotations
+
+from sanic import response
+
+
+async def s_api_metrics(request):
+    try:
+        from prometheus_client import generate_latest, CONTENT_TYPE_LATEST  # type: ignore
+        data = generate_latest()
+        return response.raw(data, content_type=CONTENT_TYPE_LATEST)
+    except Exception:
+        # Fallback: export minimal in-process counters from DHT module, if available
+        try:
+            from app.core.network.dht import prometheus as dprom
+
+            def dump(metric_obj, metric_name):
+                lines = []
+                values = getattr(metric_obj, "_values", {})
+                for labels, value in values.items():
+                    label_str = ",".join(f'{k}="{v}"' for k, v in labels)
+                    if label_str:
+                        lines.append(f"{metric_name}{{{label_str}}} {value}")
+                    else:
+                        lines.append(f"{metric_name} {value}")
+                return lines
+
+            parts = []
+            parts += dump(dprom.replication_under, "dht_replication_under_total")
+            parts += dump(dprom.replication_over, "dht_replication_over_total")
+            parts += dump(dprom.leader_changes, "dht_leader_changes_total")
+            parts += dump(dprom.merge_conflicts, "dht_merge_conflicts_total")
+            parts += dump(dprom.view_count_total, "dht_view_count_total")
+            parts += dump(dprom.unique_estimate, "dht_unique_view_estimate")
+            parts += dump(dprom.watch_time_seconds, "dht_watch_time_seconds")
+            body = "\n".join(parts) + ("\n" if parts else "")
+            return response.text(body, content_type="text/plain; version=0.0.4")
+        except Exception:
+            return response.text("")
+

app/api/routes/network.py

@@ -4,12 +4,11 @@ import json
 from datetime import datetime
 from typing import Dict, Any
 
-from base58 import b58decode
+from app.core._utils.b58 import b58decode
 from sanic import response
-from sqlalchemy import select
+from urllib.parse import urlparse
 
 from app.core.logger import make_log
-from app.core.models.my_network import KnownNode
 from app.core.network.constants import CURRENT_PROTOCOL_VERSION, NODE_TYPE_PRIVATE
 from app.core.network.config import NODE_PRIVACY
 from app.core.network.handshake import build_handshake_payload, compute_node_info, sign_response
@@ -17,6 +16,56 @@ from app.core.network.nodes import upsert_known_node, list_known_public_nodes
 from app.core.network.semver import compatibility
 from app.core.network.guard import check_rate_limit, check_timestamp_fresh, check_and_remember_nonce
 from app.core.network.config import HANDSHAKE_TS_TOLERANCE_SEC
+from app.core.ipfs_client import swarm_connect
+from app.core._config import PROJECT_HOST
+from app.core.events.service import record_event
+from app.core.network.asn import resolver as asn_resolver
+from app.core.network.dht import compute_node_id, dht_config, ReachabilityReceipt
+
+
+def _port_from_public_host(public_host: str) -> int:
+    """Return an integer port extracted from a public_host URL or host:port string."""
+    if not public_host:
+        return 80
+    parsed = urlparse(public_host)
+    if parsed.scheme:
+        if parsed.port:
+            return parsed.port
+        return 443 if parsed.scheme == "https" else 80
+    host_port = public_host.strip()
+    if ":" in host_port:
+        candidate = host_port.rsplit(":", 1)[-1]
+        try:
+            return int(candidate)
+        except (TypeError, ValueError):
+            pass
+    return 80
+
+
+def _extract_ipfs_meta(payload: Dict[str, Any]) -> Dict[str, Any]:
+    ipfs = payload or {}
+    multiaddrs = ipfs.get("multiaddrs") or []
+    if not isinstance(multiaddrs, list):
+        multiaddrs = [multiaddrs]
+    normalized_multiaddrs = [str(m) for m in multiaddrs if m]
+    meta: Dict[str, Any] = {}
+    if normalized_multiaddrs:
+        meta["multiaddrs"] = normalized_multiaddrs
+    peer_id = ipfs.get("peer_id")
+    if peer_id:
+        meta["peer_id"] = str(peer_id)
+    agent = ipfs.get("agent_version") or ipfs.get("agentVersion")
+    if agent:
+        meta["agent_version"] = str(agent)
+    return meta
+
+
+async def _connect_ipfs_multiaddrs(addrs):
+    for addr in addrs or []:
+        try:
+            await swarm_connect(addr)
+        except Exception:
+            pass
 
 
 async def s_api_v1_network_info(request):
@@ -44,7 +93,7 @@ async def s_api_v1_network_handshake(request):
         return response.json({"error": "RATE_LIMIT"}, status=429)
 
     data = request.json or {}
-    required = ["version", "public_key", "node_type", "metrics", "timestamp", "signature"]
+    required = ["version", "schema_version", "public_key", "node_id", "node_type", "metrics", "timestamp", "signature"]
     for f in required:
         if f not in data:
             return response.json({"error": f"Missing field {f}"}, status=400)
@@ -60,7 +109,19 @@ async def s_api_v1_network_handshake(request):
     if not data.get("nonce") or not check_and_remember_nonce(request.app.ctx.memory, data.get("public_key"), data.get("nonce")):
         return response.json({"error": "NONCE_REPLAY"}, status=400)
 
+    # Base schema and identity checks
+    if data.get("schema_version") != dht_config.schema_version:
+        return response.json({"error": "UNSUPPORTED_SCHEMA_VERSION"}, status=400)
+
+    try:
+        expected_node_id = compute_node_id(b58decode(data["public_key"]))
+    except Exception:
+        return response.json({"error": "BAD_PUBLIC_KEY"}, status=400)
+    if data.get("node_id") != expected_node_id:
+        return response.json({"error": "NODE_ID_MISMATCH"}, status=400)
+
     peer_version = str(data.get("version"))
+    ipfs_meta = _extract_ipfs_meta(data.get("ipfs") or {})
     comp = compatibility(peer_version, CURRENT_PROTOCOL_VERSION)
     if comp == "blocked":
         # We still store the node but respond with 409
@@ -68,7 +129,7 @@ async def s_api_v1_network_handshake(request):
             await upsert_known_node(
                 request.ctx.db_session,
                 host=data.get("public_host"),
-                port=int(str(data.get("public_host") or "").split(":")[-1]) if ":" in str(data.get("public_host") or "") else 80,
+                port=_port_from_public_host(data.get("public_host")),
                 public_key=str(data.get("public_key")),
                 meta={
                     "version": peer_version,
@@ -76,6 +137,7 @@ async def s_api_v1_network_handshake(request):
                     "is_public": data.get("node_type", "public") != "private",
                     "public_host": data.get("public_host"),
                     "unsupported_last_checked_at": datetime.utcnow().isoformat(),
+                    "ipfs": ipfs_meta,
                 }
             )
         except Exception:
@@ -88,22 +150,90 @@ async def s_api_v1_network_handshake(request):
             "peer": peer_version,
         }, status=409)
 
-    # Verify signature
+    # Verify signature (Ed25519). If libsodium not available, accept but log a warning.
+    signed_fields = {k: v for (k, v) in data.items() if k != "signature"}
+    blob = json.dumps(signed_fields, sort_keys=True, separators=(",", ":")).encode()
+    ok = False
     try:
-        # Verify signature over the entire payload except the signature itself
-        signed_fields = {k: v for (k, v) in data.items() if k != "signature"}
-        blob = json.dumps(signed_fields, sort_keys=True, separators=(",", ":")).encode()
-        import nacl.signing, nacl.encoding
-        vk = nacl.signing.VerifyKey(b58decode(data["public_key"]))
-        sig = b58decode(data["signature"])
+        import nacl.signing, nacl.encoding  # type: ignore
+        vk = nacl.signing.VerifyKey(b58decode(data.get("public_key", "")))
+        sig = b58decode(data.get("signature", ""))
         vk.verify(blob, sig)
         ok = True
-    except Exception:
+    except Exception as e:
         ok = False
     if not ok:
         make_log("Handshake", f"Signature verification failed from {data.get('public_host')}", level='warning')
         return response.json({"error": "BAD_SIGNATURE"}, status=400)
 
+    # Update membership / reachability information
+    try:
+        membership_mgr = getattr(request.app.ctx.memory, "membership", None)
+        if membership_mgr:
+            remote_ip = (request.headers.get('X-Forwarded-For') or request.remote_addr or request.ip or '').split(',')[0].strip() or None
+            # Determine caller ASN using advertised value or resolver
+            remote_asn = data.get("asn")
+            if remote_asn is None:
+                remote_asn = await asn_resolver.resolve_async(remote_ip, request.ctx.db_session)
+            else:
+                if remote_ip:
+                    asn_resolver.learn(remote_ip, int(remote_asn))
+            membership_mgr.update_member(
+                node_id=data["node_id"],
+                public_key=data["public_key"],
+                ip=remote_ip,
+                asn=int(remote_asn) if remote_asn is not None else None,
+                metadata={
+                    "capabilities": data.get("capabilities", {}),
+                    "metrics": data.get("metrics", {}),
+                    "public_host": data.get("public_host"),
+                },
+            )
+            for receipt in data.get("reachability_receipts") or []:
+                if not receipt.get("target_id") or not receipt.get("issuer_id"):
+                    continue
+                try:
+                    # Only accept receipts issued by the caller
+                    issuer_id = str(receipt.get("issuer_id"))
+                    if issuer_id != data["node_id"]:
+                        continue
+                    # Canonical message for receipt verification
+                    # schema_version is embedded to avoid replay across versions
+                    rec_asn = receipt.get("asn")
+                    if rec_asn is None:
+                        rec_asn = remote_asn
+                    payload = {
+                        "schema_version": dht_config.schema_version,
+                        "target_id": str(receipt.get("target_id")),
+                        "issuer_id": issuer_id,
+                        "asn": int(rec_asn) if rec_asn is not None else None,
+                        "timestamp": float(receipt.get("timestamp", data.get("timestamp"))),
+                    }
+                    blob = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()
+                    try:
+                        import nacl.signing  # type: ignore
+                        from app.core._utils.b58 import b58decode as _b58d
+                        vk = nacl.signing.VerifyKey(_b58d(data["public_key"]))
+                        sig_b = _b58d(str(receipt.get("signature", "")))
+                        vk.verify(blob, sig_b)
+                        # Accept and persist
+                        membership_mgr.record_receipt(
+                            ReachabilityReceipt(
+                                target_id=payload["target_id"],
+                                issuer_id=payload["issuer_id"],
+                                asn=payload["asn"],
+                                timestamp=payload["timestamp"],
+                                signature=str(receipt.get("signature", "")),
+                            )
+                        )
+                    except Exception:
+                        # Ignore invalid receipts
+                        continue
+                except Exception:
+                    continue
+    except Exception as exc:
+        make_log("Handshake", f"Membership ingest failed: {exc}", level='warning')
+
     # Upsert node and respond with our info + known public nodes
     # Do not persist private peers (ephemeral)
     if data.get("node_type") != "private" and data.get("public_host"):
@@ -111,7 +241,7 @@ async def s_api_v1_network_handshake(request):
             await upsert_known_node(
                 request.ctx.db_session,
                 host=data.get("public_host"),
-                port=int(str(data.get("public_host") or "").split(":")[-1]) if ":" in str(data.get("public_host") or "") else 80,
+                port=_port_from_public_host(data.get("public_host")),
                 public_key=str(data.get("public_key")),
                 meta={
                     "version": peer_version,
@@ -120,13 +250,31 @@ async def s_api_v1_network_handshake(request):
                     "public_host": data.get("public_host"),
                     "last_metrics": data.get("metrics", {}),
                     "capabilities": data.get("capabilities", {}),
+                    "ipfs": ipfs_meta,
                 }
             )
+            await _connect_ipfs_multiaddrs(ipfs_meta.get("multiaddrs"))
+            try:
+                await record_event(
+                    request.ctx.db_session,
+                    'node_registered',
+                    {
+                        'public_key': str(data.get("public_key")),
+                        'public_host': data.get("public_host"),
+                        'node_type': data.get("node_type"),
+                        'version': peer_version,
+                        'capabilities': data.get("capabilities", {}),
+                    },
+                    origin_host=PROJECT_HOST,
+                )
+            except Exception as ev_exc:
+                make_log("Events", f"Failed to record node_registered event: {ev_exc}", level="warning")
         except Exception as e:
             make_log("Handshake", f"Upsert peer failed: {e}", level='warning')
 
     # Merge advertised peers from the caller (optional field)
     for n in data.get("known_public_nodes", []) or []:
+        known_ipfs_meta = _extract_ipfs_meta(n.get("ipfs") or {})
         try:
             await upsert_known_node(
                 request.ctx.db_session,
@@ -139,17 +287,22 @@ async def s_api_v1_network_handshake(request):
                     "is_public": True,
                     "public_host": n.get("public_host") or n.get("host"),
                     "capabilities": n.get("capabilities") or {},
+                    "ipfs": known_ipfs_meta,
                 }
             )
+            await _connect_ipfs_multiaddrs(known_ipfs_meta.get("multiaddrs"))
         except Exception:
             pass
 
     node = await compute_node_info(request.ctx.db_session)
     known = await list_known_public_nodes(request.ctx.db_session)
+    membership_mgr = getattr(request.app.ctx.memory, "membership", None)
+    n_estimate = membership_mgr.n_estimate() if membership_mgr else 0
     resp = sign_response({
         "compatibility": comp,
         "node": node,
         "known_public_nodes": known,
+        "n_estimate": n_estimate,
     })
     make_log("Handshake", f"OK with {data.get('public_host')} compat={comp}")
     status = 200

app/api/routes/network_events.py

@@ -0,0 +1,77 @@
+from __future__ import annotations
+
+from typing import Dict, Any
+
+from sanic import response
+from sqlalchemy import select
+
+from app.core.logger import make_log
+from app.core.models import NodeEvent, KnownNode
+from app.core.network.nodesig import verify_request
+from app.core.network.guard import check_rate_limit
+from app.core._config import PROJECT_HOST
+from app.core.events.service import LOCAL_PUBLIC_KEY
+
+
+def _origin_host() -> str | None:
+    return PROJECT_HOST.rstrip('/') if PROJECT_HOST else None
+
+
+async def s_api_v1_network_events(request):
+    remote_ip = (request.headers.get('X-Forwarded-For') or request.remote_addr or request.ip or '').split(',')[0].strip()
+    if not check_rate_limit(request.app.ctx.memory, remote_ip):
+        return response.json({"error": "RATE_LIMIT"}, status=429)
+
+    ok, node_id, reason = verify_request(request, request.app.ctx.memory)
+    if not ok:
+        return response.json({"error": reason or "UNAUTHORIZED"}, status=401)
+
+    session = request.ctx.db_session
+    trusted = (await session.execute(
+        select(KnownNode).where(KnownNode.public_key == node_id)
+    )).scalar_one_or_none()
+    role = (trusted.meta or {}).get('role') if trusted and trusted.meta else None
+    if role != 'trusted':
+        make_log("Events", f"Rejected events fetch from non-trusted node {node_id}", level="warning")
+        return response.json({"error": "FORBIDDEN"}, status=403)
+
+    try:
+        since = int(request.args.get('since') or 0)
+    except (TypeError, ValueError):
+        since = 0
+    since = max(since, 0)
+
+    try:
+        limit = int(request.args.get('limit') or 100)
+    except (TypeError, ValueError):
+        limit = 100
+    limit = max(1, min(limit, 200))
+
+    result = await session.execute(
+        select(NodeEvent)
+        .where(NodeEvent.origin_public_key == LOCAL_PUBLIC_KEY, NodeEvent.seq > since)
+        .order_by(NodeEvent.seq.asc())
+        .limit(limit)
+    )
+    rows = result.scalars().all()
+
+    events: list[Dict[str, Any]] = []
+    next_since = since
+    for row in rows:
+        next_since = max(next_since, int(row.seq))
+        events.append({
+            "origin_public_key": row.origin_public_key,
+            "origin_host": row.origin_host or _origin_host(),
+            "seq": int(row.seq),
+            "uid": row.uid,
+            "event_type": row.event_type,
+            "payload": row.payload,
+            "signature": row.signature,
+            "created_at": (row.created_at.isoformat() + 'Z') if row.created_at else None,
+        })
+
+    payload = {
+        "events": events,
+        "next_since": next_since,
+    }
+    return response.json(payload)

app/api/routes/progressive_storage.py

@@ -16,6 +16,14 @@ from app.core.models.node_storage import StoredContent
 from app.core._config import UPLOADS_DIR
 from app.core.models.content_v3 import ContentDerivative
 from app.core._utils.resolve_content import resolve_content
+from app.core.network.nodesig import verify_request
+from app.core.models.my_network import KnownNode
+from sqlalchemy import select as sa_select
+import httpx
+from app.core._crypto.signer import Signer
+from app.core._secrets import hot_seed
+from app.core._utils.b58 import b58encode as _b58e, b58decode as _b58d
+import json, time
 
 
 # POST /api/v1.5/storage
@@ -305,3 +313,125 @@ async def s_api_v1_5_storage_get(request, file_hash):
     else:
         make_log("uploader_v1.5", f"Returning full file for video/audio: {final_path}", level="INFO")
         return await response.file(final_path, mime_type=mime_type)
+
+
+# GET /api/v1/storage.fetch/<file_hash>
+# Внутренний эндпойнт для межузлового запроса (NodeSig). Возвращает файл, если он есть локально.
+async def s_api_v1_storage_fetch(request, file_hash):
+    ok, node_id, reason = verify_request(request, request.app.ctx.memory)
+    if not ok:
+        return response.json({"error": reason or "UNAUTHORIZED"}, status=401)
+    # Только доверенные узлы
+    try:
+        session = request.ctx.db_session
+        row = (await session.execute(sa_select(KnownNode).where(KnownNode.public_key == node_id))).scalars().first()
+        role = (row.meta or {}).get('role') if row and row.meta else None
+        if role != 'trusted':
+            return response.json({"error": "DENIED_NOT_TRUSTED"}, status=403)
+    except Exception:
+        pass
+    # Переиспользуем реализацию v1.5
+    return await s_api_v1_5_storage_get(request, file_hash)
+
+
+# GET /api/v1/storage.proxy/<file_hash>
+# Проксирование для web-клиента: если локально нет файла, попытка получить у доверенных узлов по NodeSig
+async def s_api_v1_storage_proxy(request, file_hash):
+    # Require either valid NodeSig (unlikely for public clients) or a signed access token
+    # Token fields: pub, exp, scope, uid, sig over json {hash,scope,exp,uid}
+    def _verify_access_token() -> bool:
+        try:
+            pub = (request.args.get('pub') or '').strip()
+            exp = int(request.args.get('exp') or '0')
+            scope = (request.args.get('scope') or '').strip()
+            uid = int(request.args.get('uid') or '0')
+            sig = (request.args.get('sig') or '').strip()
+            if not pub or not exp or not scope or not sig:
+                return False
+            if exp < int(time.time()):
+                return False
+            payload = {
+                'hash': file_hash,
+                'scope': scope,
+                'exp': exp,
+                'uid': uid,
+            }
+            blob = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()
+            import nacl.signing
+            vk = nacl.signing.VerifyKey(_b58d(pub))
+            vk.verify(blob, _b58d(sig))
+            # Note: we do not require a session-bound user for media fetches,
+            # the short‑lived signature itself is sufficient.
+            return True
+        except Exception:
+            return False
+
+    ok_nodesig, _nid, _reason = verify_request(request, request.app.ctx.memory)
+    if not ok_nodesig and not _verify_access_token():
+        return response.json({'error': 'UNAUTHORIZED'}, status=401)
+    # Сначала пробуем локально без возврата 404
+    try:
+        from base58 import b58encode as _b58e
+        try:
+            # Поддержка как хэша, так и CID
+            from app.core._utils.resolve_content import resolve_content as _res
+            cid, _ = _res(file_hash)
+            file_hash = _b58e(cid.content_hash).decode()
+        except Exception:
+            pass
+        final_path = os.path.join(UPLOADS_DIR, f"{file_hash}")
+        if os.path.exists(final_path):
+            return await s_api_v1_5_storage_get(request, file_hash)
+    except Exception:
+        pass
+    # Локально нет — пробуем у доверенных
+    try:
+        async with request.app.ctx.memory.transaction("storage.proxy"):
+            # Соберём список trusted узлов
+            session = request.ctx.db_session
+            nodes = (await session.execute(sa_select(KnownNode))).scalars().all()
+            candidates = []
+            for n in nodes:
+                role = (n.meta or {}).get('role') if n.meta else None
+                if role != 'trusted':
+                    continue
+                host = (n.meta or {}).get('public_host') or (n.ip or '')
+                if not host:
+                    continue
+                base = host.rstrip('/')
+                if not base.startswith('http'):
+                    base = f"http://{base}:{n.port or 80}"
+                candidates.append(base)
+            # Проксируем с передачей Range, стриминг
+            range_header = request.headers.get("Range")
+            timeout = httpx.Timeout(10.0, read=60.0)
+            for base in candidates:
+                url = f"{base}/api/v1/storage.fetch/{file_hash}"
+                try:
+                    # Подпишем NodeSig
+                    from app.core._secrets import hot_seed, hot_pubkey
+                    from app.core.network.nodesig import sign_headers
+                    from app.core._utils.b58 import b58encode as _b58e
+                    pk_b58 = _b58e(hot_pubkey).decode()
+                    headers = sign_headers('GET', f"/api/v1/storage.fetch/{file_hash}", b"", hot_seed, pk_b58)
+                    if range_header:
+                        headers['Range'] = range_header
+                    async with httpx.AsyncClient(timeout=timeout) as client:
+                        r = await client.get(url, headers=headers)
+                        if r.status_code == 404:
+                            continue
+                        if r.status_code not in (200, 206):
+                            continue
+                        # Проксируем заголовки контента
+                        resp = await request.respond(status=r.status_code, headers={
+                            k: v for k, v in r.headers.items() if k.lower() in ("content-type", "content-length", "content-range", "accept-ranges")
+                        })
+                        async for chunk in r.aiter_bytes(chunk_size=1024*1024):
+                            await resp.send(chunk)
+                        await resp.eof()
+                        return resp
+                except Exception as e:
+                    continue
+    except Exception:
+        pass
+    return response.json({"error": "File not found"}, status=404)

app/api/routes/upload_status.py

@@ -27,9 +27,12 @@ async def s_api_v1_upload_status(request, upload_id: str):
                 {"kind": kind, "status": status}
                 for kind, status in derivative_rows
             ]
-            required = {"decrypted_high", "decrypted_low"}
-            if ec.preview_enabled and ec.content_type.startswith("video/"):
-                required.add("decrypted_preview")
+            if ec.content_type and ec.content_type.startswith("audio/"):
+                required = {"decrypted_high", "decrypted_low"}
+            elif ec.content_type and ec.content_type.startswith("video/"):
+                required = {"decrypted_high", "decrypted_low", "decrypted_preview"}
+            else:
+                required = {"decrypted_original"}
             statuses = {kind: status for kind, status in derivative_rows}
             if required and all(statuses.get(k) == "ready" for k in required):
                 conv_state = "ready"

app/api/routes/upload_tus.py

@@ -9,8 +9,9 @@ from typing import Dict, Any
 import aiofiles
 from base58 import b58encode
 from sanic import response
+import magic  # type: ignore
 
-from app.core._config import UPLOADS_DIR
+from app.core._config import UPLOADS_DIR, PROJECT_HOST
 from app.core._secrets import hot_pubkey
 from app.core.crypto.aes_gcm_stream import encrypt_file_to_encf, CHUNK_BYTES
 from app.core.crypto.keywrap import wrap_dek, KeyWrapError
@@ -20,6 +21,7 @@ from app.core.models.content_v3 import EncryptedContent, ContentKey, IpfsSync, C
 from app.core.models.node_storage import StoredContent
 from app.core.storage import db_session
 from app.core._utils.resolve_content import resolve_content
+from app.core.events.service import record_event
 from sqlalchemy import select
 
 
@@ -69,9 +71,40 @@ async def s_api_v1_upload_tus_hook(request):
     meta = upload.get("MetaData") or {}
     # Common metadata keys
     title = meta.get("title") or meta.get("Title") or meta.get("name") or "Untitled"
+    artist = (meta.get("artist") or meta.get("Artist") or "").strip()
     description = meta.get("description") or meta.get("Description") or ""
     content_type = meta.get("content_type") or meta.get("Content-Type") or "application/octet-stream"
-    preview_enabled = content_type.startswith("audio/") or content_type.startswith("video/")
+    detected_content_type = None
+    try:
+        raw_detected = magic.from_file(file_path, mime=True)
+        if raw_detected:
+            detected_content_type = raw_detected.split(";")[0].strip()
+    except Exception as e:
+        make_log("tus-hook", f"magic MIME detection failed for {file_path}: {e}", level="warning")
+
+    def _is_av(mime: str | None) -> bool:
+        if not mime:
+            return False
+        return mime.startswith("audio/") or mime.startswith("video/")
+
+    if detected_content_type:
+        if not _is_av(detected_content_type):
+            if content_type != detected_content_type:
+                make_log(
+                    "tus-hook",
+                    f"Overriding declared content_type '{content_type}' with detected '{detected_content_type}' (binary upload)",
+                    level="info",
+                )
+            content_type = detected_content_type
+        elif not _is_av(content_type):
+            make_log(
+                "tus-hook",
+                f"Detected audio/video MIME '{detected_content_type}' replacing non-AV declaration '{content_type}'",
+                level="info",
+            )
+            content_type = detected_content_type
+
+    preview_enabled = _is_av(content_type)
     # Optional preview window overrides from tus metadata
     try:
         start_ms = int(meta.get("preview_start_ms") or 0)
@@ -155,6 +188,7 @@ async def s_api_v1_upload_tus_hook(request):
         ec = EncryptedContent(
             encrypted_cid=encrypted_cid,
             title=title,
+            artist=artist or None,
             description=description,
             content_type=content_type,
             enc_size_bytes=enc_size,
@@ -196,7 +230,9 @@ async def s_api_v1_upload_tus_hook(request):
                 'storage': 'ipfs',
                 'encrypted_cid': encrypted_cid,
                 'upload_id': upload_id,
-                'source': 'tusd'
+                'source': 'tusd',
+                'title': title,
+                'artist': artist or None,
             }
             encrypted_stored_content = StoredContent(
                 type="local/encrypted_ipfs",
@@ -218,12 +254,14 @@ async def s_api_v1_upload_tus_hook(request):
             "encrypted_cid": encrypted_cid,
             "title": title,
             "description": description,
+            "artist": artist,
             "content_type": content_type,
             "size_bytes": enc_size,
             "preview_enabled": preview_enabled,
             "preview_conf": ec.preview_conf,
             "issuer_node_id": key_fpr,
             "salt_b64": _b64(salt),
+            "artist": artist or None,
         }
         try:
             from app.core._crypto.signer import Signer
@@ -235,6 +273,25 @@ async def s_api_v1_upload_tus_hook(request):
             sig = ""
         session.add(ContentIndexItem(encrypted_cid=encrypted_cid, payload=item, sig=sig))
 
+        try:
+            await record_event(
+                session,
+                'content_uploaded',
+                {
+                    'encrypted_cid': encrypted_cid,
+                    'content_hash': encrypted_hash_b58,
+                    'title': title,
+                    'description': description,
+                    'content_type': content_type,
+                    'size_bytes': enc_size,
+                    'user_id': request.ctx.user.id if getattr(request.ctx, 'user', None) else None,
+                    'telegram_id': getattr(getattr(request.ctx, 'user', None), 'telegram_id', None),
+                },
+                origin_host=PROJECT_HOST,
+            )
+        except Exception as exc:
+            make_log("Events", f"Failed to record content_uploaded event: {exc}", level="warning")
+
         await session.commit()
 
     # Update upload session with result and purge staging to avoid duplicates

app/bot/__init__.py

@@ -7,6 +7,9 @@ from app.bot.middleware import UserDataMiddleware
 from app.bot.routers.index import main_router
 
 
-dp = Dispatcher(storage=MemoryStorage())
-dp.update.outer_middleware(UserDataMiddleware())
-dp.include_router(main_router)
+def create_dispatcher() -> Dispatcher:
+    """Create aiogram Dispatcher lazily to avoid event loop issues at import time."""
+    dp = Dispatcher(storage=MemoryStorage())
+    dp.update.outer_middleware(UserDataMiddleware())
+    dp.include_router(main_router)
+    return dp

app/bot/routers/content.py

@@ -58,9 +58,12 @@ async def _compute_content_status(db_session, encrypted_cid: Optional[str], fall
             'updated_at': (row.last_access_at or row.created_at).isoformat() + 'Z' if (row.last_access_at or row.created_at) else None,
         })
 
-    required = {'decrypted_low', 'decrypted_high'}
-    if content_type.startswith('video/'):
-        required.add('decrypted_preview')
+    if content_type.startswith('audio/'):
+        required = {'decrypted_low', 'decrypted_high'}
+    elif content_type.startswith('video/'):
+        required = {'decrypted_low', 'decrypted_high', 'decrypted_preview'}
+    else:
+        required = {'decrypted_original'}
 
     statuses_by_kind = {kind: derivative_latest[kind].status for kind in required if kind in derivative_latest}
     conversion_state = 'pending'

app/bot/routers/home.py

@@ -7,6 +7,7 @@ from sqlalchemy import select, and_
 from app.core._keyboards import get_inline_keyboard
 from app.core._utils.tg_process_template import tg_process_template
 from app.core.models.wallet_connection import WalletConnection
+from app.core._config import PROJECT_HOST
 
 main_router = Router()
 
@@ -83,6 +84,35 @@ async def t_home_menu(__msg, **extra):
     return await send_home_menu(chat_wrap, user, wallet_connection, message_id=message_id)
 
 
+async def t_admin_panel(message: types.Message, **extra):
+    user = extra.get('user')
+    chat_wrap = extra.get('chat_wrap')
+    admin_host = (PROJECT_HOST or '').rstrip('/')
+    if not user or not getattr(user, 'is_admin', False):
+        await chat_wrap.send_message("Доступ к админ-панели ограничен.")
+        return
+    if not admin_host:
+        await chat_wrap.send_message("Адрес админ-панели не настроен на этой ноде.")
+        return
+    admin_url = f"{admin_host}/admin"
+    buttons = []
+    if admin_url.startswith('https://'):
+        buttons.append({
+            'text': 'Открыть в Telegram',
+            'web_app': types.WebAppInfo(url=admin_url),
+        })
+    buttons.append({
+        'text': 'Открыть в браузере',
+        'url': admin_url,
+    })
+    keyboard = get_inline_keyboard([buttons]) if buttons else None
+    await chat_wrap.send_message(
+        "Админ-панель доступна по кнопке ниже.",
+        keyboard=keyboard,
+    )
+
+
 main_router.message.register(t_home_menu, Command('start'))
+main_router.message.register(t_admin_panel, Command('admin'))
 main_router.callback_query.register(t_home_menu, F.data == 'home')
 router = main_router

app/client_bot/__init__.py

@@ -6,6 +6,9 @@ from aiogram.fsm.storage.memory import MemoryStorage
 from app.bot.middleware import UserDataMiddleware
 from app.client_bot.routers.index import main_router
 
-dp = Dispatcher(storage=MemoryStorage())
-dp.update.outer_middleware(UserDataMiddleware())
-dp.include_router(main_router)
+
+def create_dispatcher() -> Dispatcher:
+    dp = Dispatcher(storage=MemoryStorage())
+    dp.update.outer_middleware(UserDataMiddleware())
+    dp.include_router(main_router)
+    return dp

app/client_bot/routers/home.py

@@ -9,6 +9,7 @@ from app.core._utils.tg_process_template import tg_process_template
 from app.core.logger import make_log
 from app.core.models.wallet_connection import WalletConnection
 from app.core.models.node_storage import StoredContent
+from app.core._config import PROJECT_HOST
 
 main_router = Router()
 
@@ -95,6 +96,35 @@ async def t_home_menu(__msg, **extra):
     return await send_home_menu(chat_wrap, user, wallet_connection, message_id=message_id)
 
 
+async def t_admin_panel(message: types.Message, **extra):
+    user = extra.get('user')
+    chat_wrap = extra.get('chat_wrap')
+    admin_host = (PROJECT_HOST or '').rstrip('/')
+    if not user or not getattr(user, 'is_admin', False):
+        await chat_wrap.send_message("Доступ к админ-панели ограничен.")
+        return
+    if not admin_host:
+        await chat_wrap.send_message("Адрес админ-панели не настроен на этой ноде.")
+        return
+    admin_url = f"{admin_host}/admin"
+    buttons = []
+    if admin_url.startswith('https://'):
+        buttons.append({
+            'text': 'Открыть в Telegram',
+            'web_app': types.WebAppInfo(url=admin_url),
+        })
+    buttons.append({
+        'text': 'Открыть в браузере',
+        'url': admin_url,
+    })
+    keyboard = get_inline_keyboard([buttons]) if buttons else None
+    await chat_wrap.send_message(
+        "Админ-панель доступна по кнопке ниже.",
+        keyboard=keyboard,
+    )
+
+
 main_router.message.register(t_home_menu, Command('start'))
+main_router.message.register(t_admin_panel, Command('admin'))
 main_router.callback_query.register(t_home_menu, F.data == 'home')
 router = main_router

app/core/_config.py

@@ -56,7 +56,7 @@ _now_str = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
 LOG_FILEPATH = f"{LOG_DIR}/{_now_str}.log"
 
 WEB_APP_URLS = {
-    'uploadContent': f"https://my-public-node-8.projscale.dev/uploadContent"
+    'uploadContent': f"https://my-public-node-103.projscale.dev/uploadContent"
 }
 
 ALLOWED_CONTENT_TYPES = [

app/core/_crypto/signer.py

@@ -1,24 +1,58 @@
-import base58
-import nacl.encoding
-import nacl.signing
+from app.core._utils.b58 import b58encode, b58decode
+
+try:
+    import nacl.encoding
+    import nacl.signing
+    import nacl.exceptions
+    _HAS_NACL = True
+except Exception:  # pragma: no cover - fallback path
+    _HAS_NACL = False
+
+from app.core._utils.hash import blake3_digest
 
 
-class Signer:
-    def __init__(self, seed: bytes):
-        if len(seed) != 32:
-            raise ValueError("Seed must be 32 bytes")
-        self.signing_key = nacl.signing.SigningKey(seed)
-        self.verify_key = self.signing_key.verify_key
+if _HAS_NACL:
 
-    def sign(self, data_bytes: bytes) -> str:
-        signed_message = self.signing_key.sign(data_bytes)
-        signature = signed_message.signature
-        return base58.b58encode(signature).decode()
+    class Signer:
+        def __init__(self, seed: bytes):
+            if len(seed) != 32:
+                raise ValueError("Seed must be 32 bytes")
+            self.signing_key = nacl.signing.SigningKey(seed)
+            self.verify_key = self.signing_key.verify_key
 
-    def verify(self, data_bytes: bytes, signature: str) -> bool:
-        signature_bytes = base58.b58decode(signature)
-        try:
-            self.verify_key.verify(data_bytes, signature_bytes)
-            return True
-        except nacl.exceptions.BadSignatureError:
-            return False
+        def sign(self, data_bytes: bytes) -> str:
+            signed_message = self.signing_key.sign(data_bytes)
+            signature = signed_message.signature
+            return b58encode(signature).decode()
+
+        def verify(self, data_bytes: bytes, signature: str) -> bool:
+            signature_bytes = b58decode(signature)
+            try:
+                self.verify_key.verify(data_bytes, signature_bytes)
+                return True
+            except nacl.exceptions.BadSignatureError:
+                return False
+
+else:
+
+    class _VerifyKey:
+        def __init__(self, key_bytes: bytes):
+            self._key_bytes = key_bytes
+
+        def encode(self) -> bytes:
+            return self._key_bytes
+
+    class Signer:
+        def __init__(self, seed: bytes):
+            if len(seed) != 32:
+                raise ValueError("Seed must be 32 bytes")
+            self.seed = seed
+            self.verify_key = _VerifyKey(seed)
+
+        def sign(self, data_bytes: bytes) -> str:
+            digest = blake3_digest(self.seed + data_bytes)
+            return b58encode(digest).decode()
+
+        def verify(self, data_bytes: bytes, signature: str) -> bool:
+            expected = self.sign(data_bytes)
+            return expected == signature

app/core/_utils/b58.py

@@ -0,0 +1,51 @@
+from __future__ import annotations
+
+try:
+    # Prefer external package if available
+    from base58 import b58encode, b58decode  # type: ignore
+except Exception:
+    # Minimal fallback (compatible subset)
+    ALPHABET = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz"
+    ALPHABET_INDEX = {c: i for i, c in enumerate(ALPHABET)}
+
+    def _to_bytes(value: bytes | bytearray | str) -> bytes:
+        if isinstance(value, (bytes, bytearray)):
+            return bytes(value)
+        if isinstance(value, str):
+            return value.encode()
+        raise TypeError("value must be bytes or str")
+
+    def b58encode(data: bytes | bytearray | str) -> bytes:
+        data = _to_bytes(data)
+        if not data:
+            return b""
+        n = int.from_bytes(data, "big")
+        out = []
+        while n > 0:
+            n, rem = divmod(n, 58)
+            out.append(ALPHABET[rem])
+        enc = "".join(reversed(out))
+        leading = 0
+        for b in data:
+            if b == 0:
+                leading += 1
+            else:
+                break
+        return ("1" * leading + enc).encode()
+
+    def b58decode(data: bytes | bytearray | str) -> bytes:
+        data_b = _to_bytes(data)
+        if not data_b:
+            return b""
+        num = 0
+        for ch in data_b.decode():
+            num = num * 58 + ALPHABET_INDEX[ch]
+        full = num.to_bytes((num.bit_length() + 7) // 8, "big")
+        leading = 0
+        for ch in data_b:
+            if ch == ord('1'):
+                leading += 1
+            else:
+                break
+        return b"\x00" * leading + full
+

app/core/_utils/create_maria_tables.py

@@ -1,4 +1,6 @@
 from sqlalchemy.ext.asyncio import AsyncEngine
+from sqlalchemy import text
+
 from app.core.models import BlockchainTask
 from app.core.models.base import AlchemyBase
 
@@ -9,4 +11,36 @@ async def create_db_tables(engine: AsyncEngine):
     BlockchainTask()
     async with engine.begin() as conn:
         await conn.run_sync(AlchemyBase.metadata.create_all)
+        await conn.execute(text("""
+            ALTER TABLE users
+            ADD COLUMN IF NOT EXISTS is_admin BOOLEAN DEFAULT FALSE
+        """))
+        await conn.execute(text("""
+            ALTER TABLE stars_invoices
+            ADD COLUMN IF NOT EXISTS telegram_id BIGINT
+        """))
+        await conn.execute(text("""
+            ALTER TABLE stars_invoices
+            ADD COLUMN IF NOT EXISTS paid_at TIMESTAMPTZ
+        """))
+        await conn.execute(text("""
+            ALTER TABLE stars_invoices
+            ADD COLUMN IF NOT EXISTS payment_tx_id VARCHAR(256)
+        """))
+        await conn.execute(text("""
+            ALTER TABLE stars_invoices
+            ADD COLUMN IF NOT EXISTS payment_node_id VARCHAR(128)
+        """))
+        await conn.execute(text("""
+            ALTER TABLE stars_invoices
+            ADD COLUMN IF NOT EXISTS payment_node_public_host VARCHAR(256)
+        """))
+        await conn.execute(text("""
+            ALTER TABLE stars_invoices
+            ADD COLUMN IF NOT EXISTS bot_username VARCHAR(128)
+        """))
+        await conn.execute(text("""
+            ALTER TABLE stars_invoices
+            ADD COLUMN IF NOT EXISTS is_remote BOOLEAN DEFAULT FALSE
+        """))
     

app/core/_utils/hash.py

@@ -0,0 +1,29 @@
+from __future__ import annotations
+
+import hashlib
+from typing import Iterable
+
+
+def _to_bytes(data: Iterable[int] | bytes | bytearray | str) -> bytes:
+    if isinstance(data, (bytes, bytearray)):
+        return bytes(data)
+    if isinstance(data, str):
+        return data.encode()
+    return bytes(data)
+
+
+def blake3_digest(data: Iterable[int] | bytes | bytearray | str) -> bytes:
+    try:
+        from blake3 import blake3  # type: ignore
+        return blake3(_to_bytes(data)).digest()
+    except Exception:
+        return hashlib.blake2s(_to_bytes(data)).digest()
+
+
+def blake3_hex(data: Iterable[int] | bytes | bytearray | str) -> str:
+    try:
+        from blake3 import blake3  # type: ignore
+        return blake3(_to_bytes(data)).hexdigest()
+    except Exception:
+        return hashlib.blake2s(_to_bytes(data)).hexdigest()
+

app/core/background/convert_v3_service.py

@@ -8,7 +8,7 @@ from datetime import datetime
 from pathlib import Path
 from typing import List, Optional, Tuple
 
-from sqlalchemy import select
+from sqlalchemy import select, and_, or_
 
 from app.core.logger import make_log
 from app.core.storage import db_session
@@ -196,17 +196,45 @@ async def _convert_content(ec: EncryptedContent, staging: PlainStaging):
             plain_filename = f"{ec.encrypted_cid}.{input_ext}" if input_ext else ec.encrypted_cid
             async with db_session() as session:
                 existing = (await session.execute(select(StoredContent).where(StoredContent.hash == file_hash))).scalars().first()
-                if not existing:
+                if existing:
+                    sc = existing
+                    sc.type = sc.type or "local/content_bin"
+                    sc.filename = plain_filename
+                    sc.meta = {
+                        **(sc.meta or {}),
+                        'encrypted_cid': ec.encrypted_cid,
+                        'kind': 'original',
+                        'content_type': ec.content_type,
+                    }
+                    sc.updated = datetime.utcnow()
+                else:
                     sc = StoredContent(
                         type="local/content_bin",
                         hash=file_hash,
                         user_id=None,
                         filename=plain_filename,
-                        meta={'encrypted_cid': ec.encrypted_cid, 'kind': 'original'},
+                        meta={
+                            'encrypted_cid': ec.encrypted_cid,
+                            'kind': 'original',
+                            'content_type': ec.content_type,
+                        },
                         created=datetime.utcnow(),
                     )
                     session.add(sc)
                     await session.flush()
+
+                encrypted_records = (await session.execute(select(StoredContent).where(StoredContent.hash == encrypted_hash_b58))).scalars().all()
+                for encrypted_sc in encrypted_records:
+                    meta = dict(encrypted_sc.meta or {})
+                    converted = dict(meta.get('converted_content') or {})
+                    converted['original'] = file_hash
+                    meta['converted_content'] = converted
+                    if 'content_type' not in meta:
+                        meta['content_type'] = ec.content_type
+                    encrypted_sc.meta = meta
+                    encrypted_sc.decrypted_content_id = sc.id
+                    encrypted_sc.updated = datetime.utcnow()
+
                 derivative = ContentDerivative(
                     content_id=ec.id,
                     kind='decrypted_original',
@@ -341,10 +369,17 @@ async def _convert_content(ec: EncryptedContent, staging: PlainStaging):
 
 async def _pick_pending(limit: int) -> List[Tuple[EncryptedContent, PlainStaging]]:
     async with db_session() as session:
-        # Find A/V contents with preview_enabled and no ready low/low_preview derivatives yet
-        ecs = (await session.execute(select(EncryptedContent).where(
-            EncryptedContent.preview_enabled == True
-        ).order_by(EncryptedContent.created_at.desc()))).scalars().all()
+        # Include preview-enabled media and non-media content that need decrypted originals
+        non_media_filter = and_(
+            EncryptedContent.content_type.isnot(None),
+            ~EncryptedContent.content_type.like('audio/%'),
+            ~EncryptedContent.content_type.like('video/%'),
+        )
+        ecs = (await session.execute(
+            select(EncryptedContent)
+            .where(or_(EncryptedContent.preview_enabled == True, non_media_filter))
+            .order_by(EncryptedContent.created_at.desc())
+        )).scalars().all()
 
         picked: List[Tuple[EncryptedContent, PlainStaging]] = []
         for ec in ecs:
@@ -365,7 +400,12 @@ async def _pick_pending(limit: int) -> List[Tuple[EncryptedContent, PlainStaging
             # Check if derivatives already ready
             rows = (await session.execute(select(ContentDerivative).where(ContentDerivative.content_id == ec.id))).scalars().all()
             kinds_ready = {r.kind for r in rows if r.status == 'ready'}
-            required = {'decrypted_low', 'decrypted_high'} if ec.content_type.startswith('audio/') else {'decrypted_low', 'decrypted_high', 'decrypted_preview'}
+            if ec.content_type.startswith('audio/'):
+                required = {'decrypted_low', 'decrypted_high'}
+            elif ec.content_type.startswith('video/'):
+                required = {'decrypted_low', 'decrypted_high', 'decrypted_preview'}
+            else:
+                required = {'decrypted_original'}
             if required.issubset(kinds_ready):
                 continue
             # Always decrypt from IPFS using local or remote key
@@ -376,7 +416,12 @@ async def _pick_pending(limit: int) -> List[Tuple[EncryptedContent, PlainStaging
             if not staging:
                 peers = (await session.execute(select(KnownNode))).scalars().all()
                 for peer in peers:
-                    base_url = f"http://{peer.ip}:{peer.port}"
+                    meta = peer.meta or {}
+                    public_host = meta.get('public_host')
+                    if not public_host:
+                        last_resp = (meta.get('last_response') or {}).get('node', {}) if isinstance(meta, dict) else {}
+                        public_host = last_resp.get('public_host')
+                    base_url = public_host or f"http://{peer.ip}:{peer.port}"
                     dek = await request_key_from_peer(base_url, ec.encrypted_cid)
                     if not dek:
                         continue

app/core/background/event_sync_service.py

@@ -0,0 +1,152 @@
+import asyncio
+from typing import Dict, List, Optional, Tuple
+from urllib.parse import urlencode
+
+import httpx
+from sqlalchemy import select
+
+from app.core.logger import make_log
+from app.core.storage import db_session
+from app.core.models import KnownNode, NodeEvent
+from app.core.events.service import (
+    store_remote_events,
+    upsert_cursor,
+    LOCAL_PUBLIC_KEY,
+)
+from app.core.models.events import NodeEventCursor
+from app.core._secrets import hot_pubkey, hot_seed
+from app.core.network.nodesig import sign_headers
+from base58 import b58encode
+
+
+def _node_public_base(node: KnownNode) -> Optional[str]:
+    meta = node.meta or {}
+    public_host = (meta.get('public_host') or '').strip()
+    if public_host:
+        base = public_host.rstrip('/')
+        if base.startswith('http://') or base.startswith('https://'):
+            return base
+        scheme = 'https' if node.port == 443 else 'http'
+        return f"{scheme}://{base.lstrip('/')}"
+    scheme = 'https' if node.port == 443 else 'http'
+    host = (node.ip or '').strip()
+    if not host:
+        return None
+    default_port = 443 if scheme == 'https' else 80
+    if node.port and node.port != default_port:
+        return f"{scheme}://{host}:{node.port}"
+    return f"{scheme}://{host}"
+
+
+async def _fetch_events_for_node(node: KnownNode, limit: int = 100) -> Tuple[List[Dict], int]:
+    base = _node_public_base(node)
+    if not base:
+        return [], 0
+    async with db_session() as session:
+        cursor = (await session.execute(
+            select(NodeEventCursor).where(NodeEventCursor.source_public_key == node.public_key)
+        )).scalar_one_or_none()
+        since = cursor.last_seq if cursor else 0
+    query = urlencode({"since": since, "limit": limit})
+    path = f"/api/v1/network.events?{query}"
+    url = f"{base}{path}"
+    pk_b58 = b58encode(hot_pubkey).decode()
+    headers = sign_headers("GET", path, b"", hot_seed, pk_b58)
+    async with httpx.AsyncClient(timeout=20.0) as client:
+        try:
+            resp = await client.get(url, headers=headers)
+            if resp.status_code == 403:
+                make_log("Events", f"Access denied by node {node.public_key}", level="warning")
+                return [], since
+            resp.raise_for_status()
+            data = resp.json()
+        except Exception as exc:
+            make_log("Events", f"Fetch events failed from {node.public_key}: {exc}", level="debug")
+            return [], since
+    events = data.get("events") or []
+    next_since = int(data.get("next_since") or since)
+    return events, next_since
+
+
+async def _apply_event(session, event: NodeEvent):
+    if event.event_type == "stars_payment":
+        from app.core.models import StarsInvoice
+        payload = event.payload or {}
+        invoice_id = payload.get("invoice_id")
+        telegram_id = payload.get("telegram_id")
+        content_hash = payload.get("content_hash")
+        amount = payload.get("amount")
+        if not invoice_id or not telegram_id or not content_hash:
+            return
+        invoice = (await session.execute(select(StarsInvoice).where(StarsInvoice.external_id == invoice_id))).scalar_one_or_none()
+        if not invoice:
+            invoice = StarsInvoice(
+                external_id=invoice_id,
+                user_id=payload.get("user_id"),
+                type=payload.get('type') or 'access',
+                telegram_id=telegram_id,
+                amount=amount,
+                content_hash=content_hash,
+                paid=True,
+                paid_at=event.created_at,
+                payment_node_id=payload.get("payment_node", {}).get("public_key"),
+                payment_node_public_host=payload.get("payment_node", {}).get("public_host"),
+                bot_username=payload.get("bot_username"),
+                is_remote=True,
+            )
+            session.add(invoice)
+        else:
+            invoice.paid = True
+            invoice.paid_at = invoice.paid_at or event.created_at
+            invoice.payment_node_id = payload.get("payment_node", {}).get("public_key")
+            invoice.payment_node_public_host = payload.get("payment_node", {}).get("public_host")
+            invoice.bot_username = payload.get("bot_username") or invoice.bot_username
+            invoice.telegram_id = telegram_id or invoice.telegram_id
+            invoice.is_remote = invoice.is_remote or True
+            if payload.get('type'):
+                invoice.type = payload['type']
+        event.status = 'applied'
+        event.applied_at = event.applied_at or event.received_at
+    elif event.event_type == "content_indexed":
+        # The index scout will pick up via remote_content_index; we only mark event applied
+        event.status = 'recorded'
+    elif event.event_type == "node_registered":
+        event.status = 'recorded'
+    else:
+        event.status = 'recorded'
+
+
+async def main_fn(memory):
+    make_log("Events", "Sync service started", level="info")
+    while True:
+        try:
+            async with db_session() as session:
+                nodes = (await session.execute(select(KnownNode))).scalars().all()
+                trusted_nodes = [
+                    n for n in nodes
+                    if isinstance(n.meta, dict) and n.meta.get("role") == "trusted" and n.public_key != LOCAL_PUBLIC_KEY
+                ]
+                trusted_keys = {n.public_key for n in trusted_nodes}
+            for node in trusted_nodes:
+                events, next_since = await _fetch_events_for_node(node)
+                if not events:
+                    if next_since:
+                        async with db_session() as session:
+                            await upsert_cursor(session, node.public_key, next_since, node.meta.get("public_host") if isinstance(node.meta, dict) else None)
+                            await session.commit()
+                    continue
+                async with db_session() as session:
+                    stored = await store_remote_events(
+                        session,
+                        events,
+                        allowed_public_keys=trusted_keys,
+                    )
+                    for ev in stored:
+                        await _apply_event(session, ev)
+                    if stored:
+                        await session.commit()
+                    await upsert_cursor(session, node.public_key, next_since, node.meta.get("public_host") if isinstance(node.meta, dict) else None)
+                    await session.commit()
+        except Exception as exc:
+            make_log("Events", f"Sync loop error: {exc}", level="error")
+        await asyncio.sleep(10)

app/core/background/index_scout_v3.py

@@ -1,17 +1,21 @@
 import asyncio
 import os
+from datetime import datetime
 from typing import List, Optional
 
 import httpx
+from urllib.parse import urlparse
 import random
 import shutil
 from sqlalchemy import select
 
 from app.core.logger import make_log
 from app.core.storage import db_session
-from app.core.models.my_network import KnownNode
+from app.core.models.my_network import KnownNode, RemoteContentIndex
+from app.core.models.events import NodeEvent
 from app.core.models.content_v3 import EncryptedContent, ContentDerivative
-from app.core.ipfs_client import pin_add, find_providers, swarm_connect
+from app.core.ipfs_client import pin_add, pin_ls, find_providers, swarm_connect, add_streamed_file
+from app.core.events.service import LOCAL_PUBLIC_KEY
 
 
 INTERVAL_SEC = 60
@@ -28,7 +32,8 @@ async def fetch_index(base_url: str, etag: Optional[str], since: Optional[str])
         url = f"{base_url.rstrip('/')}/api/v1/content.delta" if since else f"{base_url.rstrip('/')}/api/v1/content.index"
         if etag:
             headers['If-None-Match'] = etag
-        async with httpx.AsyncClient(timeout=20) as client:
+        # follow_redirects handles peers that force HTTPS and issue 301s
+        async with httpx.AsyncClient(timeout=20, follow_redirects=True) as client:
             r = await client.get(url, headers=headers, params=params)
             if r.status_code != 200:
                 if r.status_code == 304:
@@ -103,6 +108,71 @@ async def upsert_content(item: dict):
             make_log('index_scout_v3', f"thumbnail fetch failed for {cid}: {e}", level='warning')
 
 
+def _node_base_url(node: KnownNode) -> Optional[str]:
+    meta = node.meta or {}
+    public_host = (meta.get('public_host') or '').strip()
+    if public_host:
+        base = public_host.rstrip('/')
+        if base.startswith('http://') or base.startswith('https://'):
+            return base
+        scheme = 'https' if node.port == 443 else 'http'
+        return f"{scheme}://{base.lstrip('/')}"
+    scheme = 'https' if node.port == 443 else 'http'
+    host = (node.ip or '').strip()
+    if not host:
+        return None
+    default_port = 443 if scheme == 'https' else 80
+    if node.port and node.port != default_port:
+        return f"{scheme}://{host}:{node.port}"
+    return f"{scheme}://{host}"
+
+
+async def _update_remote_index(node_id: int, items: List[dict], *, incremental: bool):
+    if not items:
+        return
+    async with db_session() as session:
+        existing_rows = (await session.execute(
+            select(RemoteContentIndex).where(RemoteContentIndex.remote_node_id == node_id)
+        )).scalars().all()
+        existing_map = {row.encrypted_hash: row for row in existing_rows if row.encrypted_hash}
+        seen = set()
+        now = datetime.utcnow()
+        for item in items:
+            cid = item.get('encrypted_cid')
+            if not cid:
+                continue
+            seen.add(cid)
+            payload_meta = {
+                'title': item.get('title'),
+                'description': item.get('description'),
+                'size_bytes': item.get('size_bytes'),
+                'preview_enabled': item.get('preview_enabled'),
+                'preview_conf': item.get('preview_conf'),
+                'issuer_node_id': item.get('issuer_node_id'),
+                'salt_b64': item.get('salt_b64'),
+            }
+            meta_clean = {k: v for k, v in payload_meta.items() if v is not None}
+            row = existing_map.get(cid)
+            if row:
+                row.content_type = item.get('content_type') or row.content_type
+                row.meta = {**(row.meta or {}), **meta_clean}
+                row.last_updated = now
+            else:
+                row = RemoteContentIndex(
+                    remote_node_id=node_id,
+                    content_type=item.get('content_type') or 'application/octet-stream',
+                    encrypted_hash=cid,
+                    meta=meta_clean,
+                    last_updated=now,
+                )
+                session.add(row)
+        if not incremental and existing_map:
+            for hash_value, row in list(existing_map.items()):
+                if hash_value not in seen:
+                    await session.delete(row)
+        await session.commit()
+
+
 async def main_fn(memory):
     make_log('index_scout_v3', 'Service started', level='info')
     sem = None
@@ -117,8 +187,70 @@ async def main_fn(memory):
                 sem = asyncio.Semaphore(max_pins)
             async with db_session() as session:
                 nodes = (await session.execute(select(KnownNode))).scalars().all()
+            node_by_pk = {n.public_key: n for n in nodes if n.public_key}
+            async with db_session() as session:
+                pending_events = (await session.execute(
+                    select(NodeEvent)
+                    .where(NodeEvent.event_type == 'content_indexed', NodeEvent.status.in_(('recorded', 'local', 'processing')))
+                    .order_by(NodeEvent.created_at.asc())
+                    .limit(25)
+                )).scalars().all()
+                for ev in pending_events:
+                    if ev.status != 'processing':
+                        ev.status = 'processing'
+                await session.commit()
+            for ev in pending_events:
+                payload = ev.payload or {}
+                cid = payload.get('encrypted_cid') or payload.get('content_cid')
+                if ev.origin_public_key == LOCAL_PUBLIC_KEY:
+                    async with db_session() as session:
+                        ref = await session.get(NodeEvent, ev.id)
+                        if ref:
+                            ref.status = 'applied'
+                            ref.applied_at = datetime.utcnow()
+                            await session.commit()
+                    continue
+                if not cid:
+                    async with db_session() as session:
+                        ref = await session.get(NodeEvent, ev.id)
+                        if ref:
+                            ref.status = 'applied'
+                            ref.applied_at = datetime.utcnow()
+                            await session.commit()
+                    continue
+                node = node_by_pk.get(ev.origin_public_key)
+                if not node:
+                    async with db_session() as session:
+                        node = (await session.execute(select(KnownNode).where(KnownNode.public_key == ev.origin_public_key))).scalar_one_or_none()
+                        if node:
+                            node_by_pk[node.public_key] = node
+                if not node:
+                    make_log('index_scout_v3', f"Event {ev.uid} refers to unknown node {ev.origin_public_key}", level='debug')
+                    async with db_session() as session:
+                        ref = await session.get(NodeEvent, ev.id)
+                        if ref:
+                            ref.status = 'recorded'
+                            await session.commit()
+                    continue
+                try:
+                    await _pin_one(node, cid)
+                    async with db_session() as session:
+                        ref = await session.get(NodeEvent, ev.id)
+                        if ref:
+                            ref.status = 'applied'
+                            ref.applied_at = datetime.utcnow()
+                            await session.commit()
+                except Exception as exc:
+                    make_log('index_scout_v3', f"Event pin failed for {cid}: {exc}", level='warning')
+                    async with db_session() as session:
+                        ref = await session.get(NodeEvent, ev.id)
+                        if ref:
+                            ref.status = 'recorded'
+                            await session.commit()
             for n in nodes:
-                base = f"http://{n.ip}:{n.port}"
+                base = _node_base_url(n)
+                if not base:
+                    continue
                 # jitter 0..30s per node to reduce stampede
                 await asyncio.sleep(random.uniform(0, 30))
                 etag = (n.meta or {}).get('index_etag')
@@ -142,6 +274,10 @@ async def main_fn(memory):
                 if not items:
                     continue
                 make_log('index_scout_v3', f"Fetched {len(items)} from {base}")
+                try:
+                    await _update_remote_index(n.id, items, incremental=bool(since))
+                except Exception as exc:
+                    make_log('index_scout_v3', f"remote index update failed for node {n.id}: {exc}", level='warning')
 
                 # Check disk watermark
                 try:
@@ -154,9 +290,23 @@ async def main_fn(memory):
                 except Exception:
                     pass
 
-                async def _pin_one(cid: str):
+                async def _pin_one(node: KnownNode, cid: str):
                     async with sem:
                         try:
+                            node_ipfs_meta = (node.meta or {}).get('ipfs') or {}
+                            multiaddrs = node_ipfs_meta.get('multiaddrs') or []
+                            for addr in multiaddrs:
+                                try:
+                                    await swarm_connect(addr)
+                                except Exception:
+                                    pass
+                            try:
+                                existing = await pin_ls(cid)
+                                if existing and existing.get('Keys'):
+                                    make_log('index_scout_v3', f"pin {cid} already present", level='debug')
+                                    return
+                            except Exception:
+                                pass
                             # Try to pre-connect to discovered providers
                             try:
                                 provs = await find_providers(cid, max_results=5)
@@ -168,16 +318,57 @@ async def main_fn(memory):
                                             pass
                             except Exception:
                                 pass
-                            await pin_add(cid, recursive=True)
+                            try:
+                                await asyncio.wait_for(pin_add(cid, recursive=True), timeout=60)
+                                return
+                            except httpx.HTTPStatusError as http_err:
+                                body = (http_err.response.text or '').lower() if http_err.response else ''
+                                if 'already pinned' in body or 'pin already set' in body:
+                                    make_log('index_scout_v3', f"pin {cid} already present", level='debug')
+                                    return
+                                raise
                         except Exception as e:
-                            make_log('index_scout_v3', f"pin {cid} failed: {e}", level='warning')
+                            # Attempt HTTP gateway fallback before logging failure
+                            fallback_sources = []
+                            node_host = node.meta.get('public_host') if isinstance(node.meta, dict) else None
+                            try:
+                                # Derive gateway host: prefer public_host domain if present
+                                parsed = urlparse(node_host) if node_host else None
+                                gateway_host = parsed.hostname if parsed and parsed.hostname else (node.ip or '').split(':')[0]
+                                gateway_port = parsed.port if (parsed and parsed.port not in (None, 80, 443)) else 8080
+                                if gateway_host:
+                                    gateway_url = f"http://{gateway_host}:{gateway_port}/ipfs/{cid}"
+                                    make_log('index_scout_v3', f"fallback download start {cid} via {gateway_url}", level='debug')
+                                    async with httpx.AsyncClient(timeout=None) as client:
+                                        resp = await client.get(gateway_url)
+                                        resp.raise_for_status()
+                                        data = resp.content
+                                    chunk_bytes = int(os.getenv('CRYPTO_CHUNK_BYTES', '1048576'))
+                                    add_params = {
+                                        'cid-version': 1,
+                                        'raw-leaves': 'true',
+                                        'chunker': f'size-{chunk_bytes}',
+                                        'hash': 'sha2-256',
+                                        'pin': 'true',
+                                    }
+                                    result = await add_streamed_file([data], filename=f'{cid}.bin', params=add_params)
+                                    if str(result.get('Hash')) != str(cid):
+                                        raise ValueError(f"gateway add returned mismatched CID {result.get('Hash')}")
+                                    make_log('index_scout_v3', f"pin {cid} fetched via gateway {gateway_host}:{gateway_port}", level='info')
+                                    return
+                                else:
+                                    fallback_sources.append('gateway-host-missing')
+                            except Exception as fallback_err:
+                                fallback_sources.append(str(fallback_err))
+                            make_log('index_scout_v3', f"pin {cid} failed: {e}; fallback={'; '.join(fallback_sources) if fallback_sources else 'none'}", level='warning')
 
                 tasks = []
                 for it in items:
                     await upsert_content(it)
                     cid = it.get('encrypted_cid')
                     if cid:
-                        tasks.append(asyncio.create_task(_pin_one(cid)))
+                        make_log('index_scout_v3', f"queue pin {cid}")
+                        tasks.append(asyncio.create_task(_pin_one(n, cid)))
                 if tasks:
                     await asyncio.gather(*tasks)
         except Exception as e:

app/core/background/indexer_service.py

@@ -8,6 +8,7 @@ from sqlalchemy import String, and_, desc, cast
 from tonsdk.boc import Cell
 from tonsdk.utils import Address
 from app.core._config import CLIENT_TELEGRAM_BOT_USERNAME, PROJECT_HOST
+from app.core.events.service import record_event
 from app.core._blockchain.ton.platform import platform
 from app.core._blockchain.ton.toncenter import toncenter
 from app.core._utils.send_status import send_status
@@ -60,45 +61,79 @@ async def indexer_loop(memory, platform_found: bool, seqno: int) -> [bool, int]:
             )
         ))).scalars().all()
         for new_license in new_licenses:
-            licensed_content = (await session.execute(select(StoredContent).where(
-                StoredContent.id == new_license.content_id
-            ))).scalars().first()
-            if not licensed_content:
-                make_log("Indexer", f"Licensed content not found: {new_license.content_id}", level="error")
+            try:
+                licensed_content = (await session.execute(select(StoredContent).where(
+                    StoredContent.id == new_license.content_id
+                ))).scalars().first()
+                if not licensed_content:
+                    make_log("Indexer", f"Licensed content not found: {new_license.content_id}", level="error")
+                    new_license.meta = {**(new_license.meta or {}), 'notification_sent': True, 'notification_error': 'content_not_found'}
+                    await session.commit()
+                    continue
 
-            content_metadata = await licensed_content.metadata_json_async(session)
-            assert content_metadata, "No content metadata found"
-
-            if not (licensed_content.owner_address == new_license.owner_address):
                 try:
-                    user = await session.get(User, new_license.user_id)
-                    if user.telegram_id and licensed_content:
-                        await (Wrapped_CBotChat(memory._client_telegram_bot, chat_id=user.telegram_id, user=user, db_session=session)).send_content(
-                            session, licensed_content
-                        )
-
-                    wallet_owner_connection = (await session.execute(
-                        select(WalletConnection).where(
-                            WalletConnection.wallet_address == licensed_content.owner_address,
-                            WalletConnection.invalidated == False
-                        ).order_by(desc(WalletConnection.id))
-                    )).scalars().first()
-                    wallet_owner_user = await session.get(User, wallet_owner_connection.user_id) if wallet_owner_connection else None
-                    if wallet_owner_user.telegram_id:
-                        wallet_owner_bot = Wrapped_CBotChat(memory._telegram_bot, chat_id=wallet_owner_user.telegram_id, user=wallet_owner_user, db_session=session)
-                        await wallet_owner_bot.send_message(
-                            user.translated('p_licenseWasBought').format(
-                                username=user.front_format(),
-                                nft_address=f'"https://tonviewer.com/{new_license.onchain_address}"',
-                                content_title=content_metadata.get('name', 'Unknown'),
-                            ),
-                            message_type='notification',
-                        )
+                    content_metadata = await licensed_content.metadata_json_async(session)
                 except BaseException as e:
-                    make_log("IndexerSendNewLicense", f"Error: {e}" + '\n' + traceback.format_exc(), level="error")
+                    make_log("Indexer", f"Metadata fetch failed for content_id={licensed_content.id}: {e}", level="warning")
+                    content_metadata = None
 
-            new_license.meta = {**new_license.meta, 'notification_sent': True}
-            await session.commit()
+                # Metadata is best-effort here: it should never block indexer loop progress.
+                if not content_metadata:
+                    content_metadata = {
+                        'name': licensed_content.meta.get('title') or licensed_content.filename or 'Unknown',
+                        'artist': licensed_content.meta.get('artist'),
+                        'title': licensed_content.meta.get('title'),
+                    }
+
+                if not (licensed_content.owner_address == new_license.owner_address):
+                    try:
+                        user = await session.get(User, new_license.user_id)
+                        if user and user.telegram_id:
+                            await (
+                                Wrapped_CBotChat(
+                                    memory._client_telegram_bot,
+                                    chat_id=user.telegram_id,
+                                    user=user,
+                                    db_session=session,
+                                )
+                            ).send_content(session, licensed_content)
+
+                        wallet_owner_connection = (await session.execute(
+                            select(WalletConnection).where(
+                                WalletConnection.wallet_address == licensed_content.owner_address,
+                                WalletConnection.invalidated == False
+                            ).order_by(desc(WalletConnection.id))
+                        )).scalars().first()
+                        wallet_owner_user = await session.get(User, wallet_owner_connection.user_id) if wallet_owner_connection else None
+                        if wallet_owner_user and wallet_owner_user.telegram_id:
+                            wallet_owner_bot = Wrapped_CBotChat(
+                                memory._telegram_bot,
+                                chat_id=wallet_owner_user.telegram_id,
+                                user=wallet_owner_user,
+                                db_session=session,
+                            )
+                            meta_title = content_metadata.get('title') or content_metadata.get('name') or 'Unknown'
+                            meta_artist = content_metadata.get('artist')
+                            formatted_title = f"{meta_artist} – {meta_title}" if meta_artist else meta_title
+                            await wallet_owner_bot.send_message(
+                                user.translated('p_licenseWasBought').format(
+                                    username=user.front_format(),
+                                    nft_address=f'"https://tonviewer.com/{new_license.onchain_address}"',
+                                    content_title=formatted_title,
+                                ),
+                                message_type='notification',
+                            )
+                    except BaseException as e:
+                        make_log("IndexerSendNewLicense", f"Error: {e}" + '\n' + traceback.format_exc(), level="error")
+
+                # Preserve current behavior: do not retry notifications indefinitely.
+                new_license.meta = {**(new_license.meta or {}), 'notification_sent': True}
+                await session.commit()
+            except BaseException as e:
+                # Never allow a single broken license/metadata record to block the whole indexer loop.
+                make_log("Indexer", f"Error processing new license {getattr(new_license, 'id', None)}: {e}" + '\n' + traceback.format_exc(), level="error")
+                new_license.meta = {**(new_license.meta or {}), 'notification_sent': True, 'notification_error': str(e)[:256]}
+                await session.commit()
 
         content_without_cid = (await session.execute(select(StoredContent).where(StoredContent.content_id == None))).scalars().all()
         for target_content in content_without_cid:
@@ -235,7 +270,20 @@ async def indexer_loop(memory, platform_found: bool, seqno: int) -> [bool, int]:
                         user = await session.get(User, user_wallet_connection.user_id)
 
                     if user:
-                        user_uploader_wrapper = Wrapped_CBotChat(memory._telegram_bot, chat_id=user.telegram_id, user=user, db_session=session)
+                        # Notify user about indexed content via client bot (main UX bot),
+                        # but keep ability to clean up uploader-bot hint messages.
+                        user_client_wrapper = Wrapped_CBotChat(
+                            memory._client_telegram_bot,
+                            chat_id=user.telegram_id,
+                            user=user,
+                            db_session=session,
+                        )
+                        user_uploader_wrapper = Wrapped_CBotChat(
+                            memory._telegram_bot,
+                            chat_id=user.telegram_id,
+                            user=user,
+                            db_session=session,
+                        )
                         ref_id = (user.meta or {}).get('ref_id')
                         if not ref_id:
                             ref_id = user.ensure_ref_id()
@@ -246,12 +294,12 @@ async def indexer_loop(memory, platform_found: bool, seqno: int) -> [bool, int]:
                             item_index=item_index,
                         )
 
-                        await user_uploader_wrapper.send_message(
+                        await user_client_wrapper.send_message(
                             message_text,
                             message_type='notification'
                         )
 
-                        await user_uploader_wrapper.send_content(
+                        await user_client_wrapper.send_content(
                             session,
                             encrypted_stored_content
                         )
@@ -266,7 +314,11 @@ async def indexer_loop(memory, platform_found: bool, seqno: int) -> [bool, int]:
                                 )
                             ))
                             for hint_message in result.scalars().all():
-                                await user_uploader_wrapper.delete_message(hint_message.message_id)
+                                # Delete the hint with the bot that originally sent it.
+                                if hint_message.bot_id == user_client_wrapper.bot_id:
+                                    await user_client_wrapper.delete_message(hint_message.message_id)
+                                elif hint_message.bot_id == user_uploader_wrapper.bot_id:
+                                    await user_uploader_wrapper.delete_message(hint_message.message_id)
                         except BaseException as e:
                             make_log("Indexer", f"Error while deleting hint messages: {e}" + '\n' + traceback.format_exc(), level="error")
                 elif encrypted_stored_content.type.startswith('onchain') and encrypted_stored_content.onchain_index == item_index:
@@ -284,6 +336,21 @@ async def indexer_loop(memory, platform_found: bool, seqno: int) -> [bool, int]:
                     **item_metadata_packed
                 }
                 encrypted_stored_content.content_id = item_content_cid_str
+                try:
+                    await record_event(
+                        session,
+                        'content_indexed',
+                        {
+                            'onchain_index': item_index,
+                            'content_hash': item_content_hash_str,
+                            'encrypted_cid': item_content_cid_str,
+                            'item_address': item_address.to_string(1, 1, 1),
+                            'owner_address': item_owner_address.to_string(1, 1, 1) if item_owner_address else None,
+                        },
+                        origin_host=PROJECT_HOST,
+                    )
+                except Exception as exc:
+                    make_log("Events", f"Failed to record content_indexed event: {exc}", level="warning")
 
                 await session.commit()
                 return platform_found, seqno
@@ -308,6 +375,21 @@ async def indexer_loop(memory, platform_found: bool, seqno: int) -> [bool, int]:
             updated=datetime.now()
         )
         session.add(onchain_stored_content)
+        try:
+            await record_event(
+                session,
+                'content_indexed',
+                {
+                    'onchain_index': item_index,
+                    'content_hash': item_content_hash_str,
+                    'encrypted_cid': item_content_cid_str,
+                    'item_address': item_address.to_string(1, 1, 1),
+                    'owner_address': item_owner_address.to_string(1, 1, 1) if item_owner_address else None,
+                },
+                origin_host=PROJECT_HOST,
+            )
+        except Exception as exc:
+            make_log("Events", f"Failed to record content_indexed event: {exc}", level="warning")
         await session.commit()
         make_log("Indexer", f"Item indexed: {item_content_hash_str}", level="info")
         last_known_index += 1

app/core/background/license_service.py

@@ -18,9 +18,12 @@ from app.core.models.wallet_connection import WalletConnection
 from app.core._keyboards import get_inline_keyboard
 from app.core.models._telegram import Wrapped_CBotChat
 from app.core.storage import db_session
-from app.core._config import CLIENT_TELEGRAM_API_KEY
+from app.core._config import CLIENT_TELEGRAM_API_KEY, CLIENT_TELEGRAM_BOT_USERNAME, PROJECT_HOST
 from app.core.models.user import User
 from app.core.models import StarsInvoice
+from app.core.events.service import record_event
+from app.core._secrets import hot_pubkey
+from base58 import b58encode
 import os
 import traceback
 
@@ -53,15 +56,42 @@ async def license_index_loop(memory, platform_found: bool, seqno: int) -> [bool,
 
                         if star_payment.amount == existing_invoice.amount:
                             if not existing_invoice.paid:
+                                user = (await session.execute(select(User).where(User.id == existing_invoice.user_id))).scalars().first()
                                 existing_invoice.paid = True
+                                existing_invoice.paid_at = datetime.utcnow()
+                                existing_invoice.telegram_id = getattr(user, 'telegram_id', None)
+                                existing_invoice.payment_tx_id = getattr(star_payment, 'id', None)
+                                existing_invoice.payment_node_id = b58encode(hot_pubkey).decode()
+                                existing_invoice.payment_node_public_host = PROJECT_HOST
+                                existing_invoice.bot_username = CLIENT_TELEGRAM_BOT_USERNAME
+                                existing_invoice.is_remote = False
+                                await record_event(
+                                    session,
+                                    'stars_payment',
+                                    {
+                                        'invoice_id': existing_invoice.external_id,
+                                        'content_hash': existing_invoice.content_hash,
+                                        'amount': existing_invoice.amount,
+                                        'user_id': existing_invoice.user_id,
+                                        'telegram_id': existing_invoice.telegram_id,
+                                        'bot_username': CLIENT_TELEGRAM_BOT_USERNAME,
+                                        'type': existing_invoice.type,
+                                        'payment_node': {
+                                            'public_key': b58encode(hot_pubkey).decode(),
+                                            'public_host': PROJECT_HOST,
+                                        },
+                                        'paid_at': existing_invoice.paid_at.isoformat() + 'Z' if existing_invoice.paid_at else None,
+                                        'payment_tx_id': existing_invoice.payment_tx_id,
+                                    },
+                                    origin_host=PROJECT_HOST,
+                                )
                                 await session.commit()
 
                                 licensed_content = (await session.execute(select(StoredContent).where(StoredContent.hash == existing_invoice.content_hash))).scalars().first()
-                                user = (await session.execute(select(User).where(User.id == existing_invoice.user_id))).scalars().first()
-
-                                await (Wrapped_CBotChat(memory._client_telegram_bot, chat_id=user.telegram_id, user=user, db_session=session)).send_content(
-                                    session, licensed_content
-                                )
+                                if user and user.telegram_id and licensed_content:
+                                    await (Wrapped_CBotChat(memory._client_telegram_bot, chat_id=user.telegram_id, user=user, db_session=session)).send_content(
+                                        session, licensed_content
+                                    )
                     except BaseException as e:
                         make_log("StarsProcessing", f"Local error: {e}" + '\n' + traceback.format_exc(), level="error")
 

app/core/content/utils.py

@@ -115,6 +115,7 @@ def _clean_text_content(text: str, is_hashtag: bool = False) -> str:
 async def create_metadata_for_item(
     db_session,
     title: str = None,
+    artist: str = None,
     cover_url: str = None,
     authors: list = None,
     hashtags: list = [],
@@ -128,6 +129,15 @@ async def create_metadata_for_item(
     cleaned_title = cleaned_title[:100].strip() # Truncate and strip after cleaning
     assert len(cleaned_title) > 3, f"Cleaned title '{cleaned_title}' (from original '{title}') is too short or became empty after cleaning."
 
+    cleaned_artist = None
+    if artist:
+        cleaned_artist = _clean_text_content(artist, is_hashtag=False)
+        cleaned_artist = cleaned_artist[:100].strip()
+        if not cleaned_artist:
+            cleaned_artist = None
+
+    display_name = f"{cleaned_artist} – {cleaned_title}" if cleaned_artist else cleaned_title
+
     # Process and clean hashtags
     processed_hashtags = []
     if hashtags and isinstance(hashtags, list):
@@ -142,17 +152,21 @@ async def create_metadata_for_item(
     processed_hashtags = list(dict.fromkeys(processed_hashtags))[:10] 
 
     item_metadata = {
-        'name': cleaned_title,
-        'attributes': [
-            # {
-            #     'trait_type': 'Artist',
-            #     'value': 'Unknown'
-            # },
-        ],
+        'name': display_name,
+        'title': cleaned_title,
+        'display_name': display_name,
         'downloadable': downloadable,
         'tags': processed_hashtags,  # New field for storing the list of cleaned hashtags
+        'attributes': [],
     }
 
+    if cleaned_artist:
+        item_metadata['artist'] = cleaned_artist
+        item_metadata['attributes'].append({
+            'trait_type': 'Artist',
+            'value': cleaned_artist,
+        })
+
     # Generate description from the processed hashtags
     item_metadata['description'] = ' '.join([f"#{h}" for h in processed_hashtags if h])
 

app/core/events/__init__.py

@@ -0,0 +1,17 @@
+from .service import (
+    record_event,
+    store_remote_events,
+    verify_event_signature,
+    next_local_seq,
+    upsert_cursor,
+    prune_events,
+)
+
+__all__ = [
+    'record_event',
+    'store_remote_events',
+    'verify_event_signature',
+    'next_local_seq',
+    'upsert_cursor',
+    'prune_events',
+]

app/core/events/service.py

@@ -0,0 +1,185 @@
+from __future__ import annotations
+
+import json
+from datetime import datetime, timedelta, timezone
+from typing import Any, Dict, Iterable, List, Optional
+from uuid import uuid4
+
+from base58 import b58decode, b58encode
+import nacl.signing
+from sqlalchemy import select, delete
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.logger import make_log
+from app.core._secrets import hot_pubkey, hot_seed
+from app.core.models import NodeEvent, NodeEventCursor
+
+
+LOCAL_PUBLIC_KEY = b58encode(hot_pubkey).decode()
+
+
+def _normalize_dt(value: Optional[datetime]) -> datetime:
+    if value is None:
+        return datetime.utcnow()
+    if value.tzinfo is not None:
+        return value.astimezone(timezone.utc).replace(tzinfo=None)
+    return value
+
+
+def _parse_iso_dt(iso_value: Optional[str]) -> datetime:
+    if not iso_value:
+        return datetime.utcnow()
+    try:
+        parsed = datetime.fromisoformat(iso_value.replace('Z', '+00:00'))
+    except Exception:
+        return datetime.utcnow()
+    return _normalize_dt(parsed)
+
+
+def _canonical_blob(data: Dict[str, Any]) -> bytes:
+    return json.dumps(data, sort_keys=True, separators=(",", ":")).encode()
+
+
+def _sign_event(blob: Dict[str, Any]) -> str:
+    signing_key = nacl.signing.SigningKey(hot_seed)
+    signature = signing_key.sign(_canonical_blob(blob)).signature
+    return b58encode(signature).decode()
+
+
+def verify_event_signature(event: Dict[str, Any]) -> bool:
+    try:
+        origin_key = event["origin_public_key"]
+        signature = event["signature"]
+        payload = {
+            "origin_public_key": origin_key,
+            "origin_host": event.get("origin_host"),
+            "seq": event["seq"],
+            "uid": event["uid"],
+            "event_type": event["event_type"],
+            "payload": event.get("payload") or {},
+            "created_at": event.get("created_at"),
+        }
+        verify_key = nacl.signing.VerifyKey(b58decode(origin_key))
+        verify_key.verify(_canonical_blob(payload), b58decode(signature))
+        return True
+    except Exception as exc:
+        make_log("Events", f"Signature validation failed: {exc}", level="warning")
+        return False
+
+
+async def next_local_seq(session: AsyncSession) -> int:
+    result = await session.execute(
+        select(NodeEvent.seq)
+        .where(NodeEvent.origin_public_key == LOCAL_PUBLIC_KEY)
+        .order_by(NodeEvent.seq.desc())
+        .limit(1)
+    )
+    row = result.scalar_one_or_none()
+    return int(row or 0) + 1
+
+
+async def record_event(
+    session: AsyncSession,
+    event_type: str,
+    payload: Dict[str, Any],
+    origin_host: Optional[str] = None,
+    created_at: Optional[datetime] = None,
+) -> NodeEvent:
+    seq = await next_local_seq(session)
+    created_dt = _normalize_dt(created_at)
+    event_body = {
+        "origin_public_key": LOCAL_PUBLIC_KEY,
+        "origin_host": origin_host,
+        "seq": seq,
+        "uid": uuid4().hex,
+        "event_type": event_type,
+        "payload": payload,
+        "created_at": created_dt.replace(tzinfo=timezone.utc).isoformat().replace('+00:00', 'Z'),
+    }
+    signature = _sign_event(event_body)
+    node_event = NodeEvent(
+        origin_public_key=LOCAL_PUBLIC_KEY,
+        origin_host=origin_host,
+        seq=seq,
+        uid=event_body["uid"],
+        event_type=event_type,
+        payload=payload,
+        signature=signature,
+        created_at=created_dt,
+        status='local',
+    )
+    session.add(node_event)
+    await session.flush()
+    make_log("Events", f"Recorded local event {event_type} seq={seq}")
+    return node_event
+
+
+async def upsert_cursor(session: AsyncSession, source_public_key: str, seq: int, host: Optional[str]):
+    existing = (await session.execute(
+        select(NodeEventCursor).where(NodeEventCursor.source_public_key == source_public_key)
+    )).scalar_one_or_none()
+    if existing:
+        if seq > existing.last_seq:
+            existing.last_seq = seq
+        if host:
+            existing.source_public_host = host
+    else:
+        cursor = NodeEventCursor(
+            source_public_key=source_public_key,
+            last_seq=seq,
+            source_public_host=host,
+        )
+        session.add(cursor)
+    await session.flush()
+
+
+async def store_remote_events(
+    session: AsyncSession,
+    events: Iterable[Dict[str, Any]],
+    allowed_public_keys: Optional[set[str]] = None,
+) -> List[NodeEvent]:
+    stored: List[NodeEvent] = []
+    for event in events:
+        if not verify_event_signature(event):
+            continue
+        origin_pk = event["origin_public_key"]
+        if allowed_public_keys is not None and origin_pk not in allowed_public_keys:
+            make_log("Events", f"Ignored event from untrusted node {origin_pk}", level="warning")
+            continue
+        seq = int(event["seq"])
+        exists = (await session.execute(
+            select(NodeEvent).where(
+                NodeEvent.origin_public_key == origin_pk,
+                NodeEvent.seq == seq,
+            )
+        )).scalar_one_or_none()
+        if exists:
+            continue
+        created_dt = _parse_iso_dt(event.get("created_at"))
+        received_dt = datetime.utcnow()
+        node_event = NodeEvent(
+            origin_public_key=origin_pk,
+            origin_host=event.get("origin_host"),
+            seq=seq,
+            uid=event["uid"],
+            event_type=event["event_type"],
+            payload=event.get("payload") or {},
+            signature=event["signature"],
+            created_at=created_dt,
+            status='recorded',
+            received_at=received_dt,
+        )
+        session.add(node_event)
+        stored.append(node_event)
+        await upsert_cursor(session, origin_pk, seq, event.get("origin_host"))
+        make_log("Events", f"Ingested remote event {event['event_type']} from {origin_pk} seq={seq}", level="debug")
+    if stored:
+        await session.flush()
+    return stored
+
+
+async def prune_events(session: AsyncSession, max_age_days: int = 90):
+    cutoff = datetime.utcnow() - timedelta(days=max_age_days)
+    await session.execute(
+        delete(NodeEvent).where(NodeEvent.created_at < cutoff)
+    )

app/core/models/__init__.py

@@ -11,6 +11,7 @@ from app.core.models.content.user_content import UserContent, UserAction
 from app.core.models._config import ServiceConfigValue, ServiceConfig
 from app.core.models.asset import Asset
 from app.core.models.my_network import KnownNode, KnownNodeIncident, RemoteContentIndex
+from app.core.models.events import NodeEvent, NodeEventCursor
 from app.core.models.promo import PromoAction
 from app.core.models.tasks import BlockchainTask
 from app.core.models.content_v3 import (

app/core/models/_telegram/templates/player.py

@@ -12,6 +12,7 @@ import urllib
 
 from app.core.models.transaction import StarsInvoice
 from app.core._utils.share_links import build_content_links
+from app.core.models.content_v3 import EncryptedContent
 
 
 class PlayerTemplates:
@@ -24,7 +25,6 @@ class PlayerTemplates:
         text = ""
         content_metadata_json = {}
         description_block = ""
-        status_hint = ""
         if content:
             assert content.type.startswith('onchain/content'), "Invalid nodeStorage content type"
             cd_log = f"Content (SHA256: {content.hash}), Encrypted: {content.encrypted}, TelegramCID: {content.telegram_cid}. "
@@ -90,22 +90,73 @@ class PlayerTemplates:
         if cover_content:
             template_kwargs['photo'] = URLInputFile(cover_content.web_url)
 
-        if not local_content:
-            status_hint = self.user.translated('p_playerContext_contentNotReady')
+        encrypted_cid_candidates = []
+        if content_meta:
+            encrypted_cid_candidates.extend([
+                content_meta.get('content_cid'),
+                content_meta.get('encrypted_cid'),
+            ])
+        if local_content and isinstance(local_content.meta, dict):
+            encrypted_cid_candidates.append(local_content.meta.get('encrypted_cid'))
+        if content and content.content_id:
+            encrypted_cid_candidates.append(content.content_id)
+
+        encrypted_content_row = None
+        if self.db_session:
+            for candidate in encrypted_cid_candidates:
+                if not candidate:
+                    continue
+                encrypted_content_row = (await self.db_session.execute(
+                    select(EncryptedContent).where(EncryptedContent.encrypted_cid == candidate)
+                )).scalars().first()
+                if encrypted_content_row:
+                    break
 
         description = (content_metadata_json.get('description') or '').strip()
+        encrypted_description = (encrypted_content_row.description or '').strip() if encrypted_content_row and encrypted_content_row.description else ''
+        if not description and encrypted_description:
+            description = encrypted_description
         if description:
             description_block = f"{description}\n"
 
-        title = content_metadata_json.get('name') or (local_content.filename if local_content else None) or (content.filename if content else None) or content.cid.serialize_v2()
-
-        status_block = f"{status_hint}\n" if status_hint else ""
+        metadata_title = content_metadata_json.get('title') or content_metadata_json.get('name')
+        if not metadata_title:
+            metadata_title = (
+                (encrypted_content_row.title if encrypted_content_row and encrypted_content_row.title else None)
+                or (local_content.filename if local_content else None)
+                or (content.filename if content else None)
+                or content.cid.serialize_v2()
+            )
+        metadata_artist = content_metadata_json.get('artist')
+        if metadata_artist in ('', None):
+            metadata_artist = None
+        if not metadata_artist:
+            encrypted_artist = getattr(encrypted_content_row, 'artist', None)
+            metadata_artist = encrypted_artist if encrypted_artist else metadata_artist
+        title = f"{metadata_artist} – {metadata_title}" if metadata_artist else metadata_title
 
         text = f"""<b>{title}</b>
-{description_block}{status_block}Этот контент был загружен в MY
+{description_block}Этот контент был загружен в MY
 \t/ p2p content market /
 <blockquote><a href="{content_share_link['url']}">🔴 «открыть в MY»</a></blockquote>"""
 
+        if self.db_session and content:
+            processing_messages = (await self.db_session.execute(
+                select(KnownTelegramMessage).where(
+                    and_(
+                        KnownTelegramMessage.type == 'content/processing',
+                        KnownTelegramMessage.chat_id == self._chat_id,
+                        KnownTelegramMessage.bot_id == self.bot_id,
+                        KnownTelegramMessage.deleted == False,
+                        KnownTelegramMessage.content_id == content.id,
+                    )
+                )
+            )).scalars().all()
+
+            if local_content and processing_messages:
+                for msg in processing_messages:
+                    await self.delete_message(msg.message_id)
+
         make_log("TG-Player", f"Send content {content_type} ({content_encoding}) to chat {self._chat_id}. {cd_log}")
         kmsgs = (await self.db_session.execute(select(KnownTelegramMessage).where(
             and_(

app/core/models/_telegram/wrapped_bot.py

@@ -82,7 +82,7 @@ class Wrapped_CBotChat(T, PlayerTemplates):
 
         return result
 
-    async def send_message(self, text: str, message_type='common', message_meta={}, **kwargs):
+    async def send_message(self, text: str, message_type='common', message_meta={}, content_id=None, **kwargs):
         assert self._chat_id, "No chat_id"
         try:
             make_log(self, f"Send message to {self._chat_id}. Text len: {len(text)}", level='debug')
@@ -93,7 +93,7 @@ class Wrapped_CBotChat(T, PlayerTemplates):
                 disable_web_page_preview=True,
                 **kwargs
             )
-            return await self.return_result(r, message_type=message_type, message_meta=message_meta)
+            return await self.return_result(r, message_type=message_type, message_meta=message_meta, content_id=content_id)
         except BaseException as e:
             make_log(self, f"Error sending message to {self._chat_id}. Error: {e}", level='warning')
             return None

app/core/models/content_v3.py

@@ -16,6 +16,7 @@ class EncryptedContent(AlchemyBase):
 
     # Public metadata
     title = Column(String(512), nullable=False)
+    artist = Column(String(512), nullable=True)
     description = Column(String(4096), nullable=True)
     content_type = Column(String(64), nullable=False)  # e.g. audio/flac, video/mp4, application/octet-stream
 

app/core/models/dht.py

@@ -0,0 +1,22 @@
+from __future__ import annotations
+
+from sqlalchemy import Column, String, Integer, Float, JSON, DateTime
+from datetime import datetime
+
+from .base import AlchemyBase
+
+
+class DHTRecordRow(AlchemyBase):
+    __tablename__ = 'dht_records'
+
+    # fingerprint = blake3(serialized key)
+    fingerprint = Column(String(128), primary_key=True)
+    key = Column(String(512), nullable=False, index=True)
+    schema_version = Column(String(16), nullable=False, default='v1')
+    logical_counter = Column(Integer, nullable=False, default=0)
+    timestamp = Column(Float, nullable=False, default=0.0)
+    node_id = Column(String(128), nullable=False)
+    signature = Column(String(512), nullable=True)
+    value = Column(JSON, nullable=False, default=dict)
+    updated_at = Column(DateTime, nullable=False, default=datetime.utcnow, onupdate=datetime.utcnow)
+

app/core/models/events.py

@@ -0,0 +1,48 @@
+from __future__ import annotations
+
+from datetime import datetime
+from sqlalchemy import (
+    Column,
+    Integer,
+    BigInteger,
+    String,
+    DateTime,
+    JSON,
+    UniqueConstraint,
+)
+
+from .base import AlchemyBase
+
+
+class NodeEvent(AlchemyBase):
+    __tablename__ = 'node_events'
+    __table_args__ = (
+        UniqueConstraint('origin_public_key', 'seq', name='uq_node_events_origin_seq'),
+        UniqueConstraint('uid', name='uq_node_events_uid'),
+    )
+
+    id = Column(Integer, autoincrement=True, primary_key=True)
+    origin_public_key = Column(String(128), nullable=False)
+    origin_host = Column(String(256), nullable=True)
+    seq = Column(BigInteger, nullable=False)
+    uid = Column(String(64), nullable=False)
+    event_type = Column(String(64), nullable=False)
+    payload = Column(JSON, nullable=False, default=dict)
+    signature = Column(String(512), nullable=False)
+    created_at = Column(DateTime, nullable=False, default=datetime.utcnow)
+    received_at = Column(DateTime, nullable=False, default=datetime.utcnow)
+    applied_at = Column(DateTime, nullable=True)
+    status = Column(String(32), nullable=False, default='recorded')
+
+
+class NodeEventCursor(AlchemyBase):
+    __tablename__ = 'node_event_cursors'
+    __table_args__ = (
+        UniqueConstraint('source_public_key', name='uq_event_cursor_source'),
+    )
+
+    id = Column(Integer, autoincrement=True, primary_key=True)
+    source_public_key = Column(String(128), nullable=False)
+    last_seq = Column(BigInteger, nullable=False, default=0)
+    updated_at = Column(DateTime, nullable=False, default=datetime.utcnow, onupdate=datetime.utcnow)
+    source_public_host = Column(String(256), nullable=True)

app/core/models/memory.py

@@ -4,9 +4,19 @@ from datetime import datetime
 from datetime import timedelta
 
 from aiogram import Bot
+from app.core._utils.b58 import b58encode
 
 from app.core._config import TELEGRAM_API_KEY, CLIENT_TELEGRAM_API_KEY
+from app.core._crypto.signer import Signer
+from app.core._secrets import hot_pubkey, hot_seed
 from app.core.logger import make_log
+from app.core.network.dht import (
+    MembershipManager,
+    ReplicationManager,
+    MetricsAggregator,
+    compute_node_id,
+)
+from app.core.network.dht.store import PersistentDHTStore
 
 
 class Memory:
@@ -46,6 +56,15 @@ class Memory:
         self._handshake_rl = {"minute": 0, "counts": {}}
         self._handshake_nonces = {}
 
+        # Decentralised storage components
+        self.node_id = compute_node_id(hot_pubkey)
+        self.signer = Signer(hot_seed)
+        self.dht_store = PersistentDHTStore(self.node_id, self.signer)
+        self.membership = MembershipManager(self.node_id, self.signer, self.dht_store)
+        self.replication = ReplicationManager(self.node_id, self.signer, self.dht_store)
+        self.metrics = MetricsAggregator(self.node_id, self.signer, self.dht_store)
+        self.membership.register_local(public_key=b58encode(hot_pubkey).decode(), ip=None, asn=None)
+
     @asynccontextmanager
     async def transaction(self, desc=""):
         make_log("Memory.transaction", f"Starting transaction; {desc}", level='debug')
@@ -80,4 +99,3 @@ class Memory:
                 make_log("Queue.add_task", f"Error when adding task to memory: {e}", level='error')
 
             self._execute_queue.append([_fn, args, kwargs])
-

app/core/models/rdap.py

@@ -0,0 +1,16 @@
+from __future__ import annotations
+
+from datetime import datetime
+from sqlalchemy import Column, String, Integer, DateTime
+
+from .base import AlchemyBase
+
+
+class RdapCache(AlchemyBase):
+    __tablename__ = 'rdap_cache'
+
+    ip = Column(String(64), primary_key=True)
+    asn = Column(Integer, nullable=True)
+    source = Column(String(64), nullable=True)
+    updated_at = Column(DateTime, nullable=False, default=datetime.utcnow, onupdate=datetime.utcnow)
+

app/core/models/transaction.py

@@ -1,4 +1,4 @@
-from sqlalchemy import Column, Integer, String, ForeignKey, DateTime, Boolean, Float
+from sqlalchemy import Column, Integer, BigInteger, String, ForeignKey, DateTime, Boolean, Float
 from sqlalchemy.orm import relationship
 from datetime import datetime
 
@@ -49,8 +49,15 @@ class StarsInvoice(AlchemyBase):
 
     user_id = Column(Integer, ForeignKey('users.id'), nullable=True)
     content_hash = Column(String(256), nullable=True)
+    telegram_id = Column(BigInteger, nullable=True)
 
     invoice_url = Column(String(256), nullable=True)
     paid = Column(Boolean, nullable=False, default=False)
+    paid_at = Column(DateTime, nullable=True)
+    payment_tx_id = Column(String(256), nullable=True)
+    payment_node_id = Column(String(128), nullable=True)
+    payment_node_public_host = Column(String(256), nullable=True)
+    bot_username = Column(String(128), nullable=True)
+    is_remote = Column(Boolean, nullable=False, default=False)
 
     created = Column(DateTime, nullable=False, default=datetime.utcnow)

app/core/models/user/__init__.py

@@ -1,5 +1,5 @@
 from datetime import datetime
-from sqlalchemy import Column, Integer, String, BigInteger, DateTime, JSON
+from sqlalchemy import Column, Integer, String, BigInteger, DateTime, JSON, Boolean
 from sqlalchemy.orm import relationship
 
 from app.core.auth_v1 import AuthenticationMixin as AuthenticationMixin_V1
@@ -23,6 +23,7 @@ class User(AlchemyBase, DisplayMixin, TranslationCore, AuthenticationMixin_V1, W
     username = Column(String(512), nullable=True)
     lang_code = Column(String(8), nullable=False, default="en")
     meta = Column(JSON, nullable=False, default=dict)
+    is_admin = Column(Boolean, nullable=False, default=False)
 
     last_use = Column(DateTime, nullable=False, default=datetime.utcnow)
     updated = Column(DateTime, nullable=False, default=datetime.utcnow)

app/core/network/asn.py

@@ -0,0 +1,94 @@
+from __future__ import annotations
+
+import ipaddress
+from dataclasses import dataclass, field
+from typing import Dict, Optional
+
+from app.core.logger import make_log
+
+
+@dataclass
+class ASNResolver:
+    cache: Dict[str, int] = field(default_factory=dict)
+
+    def normalise(self, ip: str | None) -> Optional[str]:
+        if not ip:
+            return None
+        try:
+            return str(ipaddress.ip_address(ip))
+        except Exception:
+            return None
+
+    def resolve(self, ip: str | None) -> Optional[int]:
+        norm = self.normalise(ip)
+        if not norm:
+            return None
+        return self.cache.get(norm)
+
+    def learn(self, ip: str, asn: int) -> None:
+        norm = self.normalise(ip)
+        if not norm:
+            make_log("ASNResolver", f"Invalid IP provided for learn: {ip}", level="warning")
+            return
+        self.cache[norm] = asn
+
+    async def resolve_async(self, ip: str | None, db_session=None) -> Optional[int]:
+        """Resolve ASN via persistent cache; fallback to RDAP API; store result.
+
+        - Checks in-memory cache first.
+        - If not found, checks DB table rdap_cache when available.
+        - If still not found, queries a public API and persists.
+        """
+        norm = self.normalise(ip)
+        if not norm:
+            return None
+        # In-memory cache first
+        if norm in self.cache:
+            return self.cache[norm]
+        # DB lookup if possible
+        try:
+            if db_session is not None:
+                from sqlalchemy import select
+                from app.core.models.rdap import RdapCache
+                row = (await db_session.execute(select(RdapCache).where(RdapCache.ip == norm))).scalars().first()
+                if row and row.asn is not None:
+                    self.cache[norm] = int(row.asn)
+                    return int(row.asn)
+        except Exception as e:
+            make_log("ASNResolver", f"DB lookup failed for {norm}: {e}", level="warning")
+
+        # Remote lookup (best-effort)
+        asn: Optional[int] = None
+        try:
+            import httpx
+            url = f"https://api.iptoasn.com/v1/as/ip/{norm}"
+            async with httpx.AsyncClient(timeout=5.0) as client:
+                r = await client.get(url)
+                if r.status_code == 200:
+                    j = r.json()
+                    num = j.get("as_number")
+                    if isinstance(num, int) and num > 0:
+                        asn = num
+        except Exception as e:
+            make_log("ASNResolver", f"RDAP lookup failed for {norm}: {e}", level="warning")
+
+        if asn is not None:
+            self.cache[norm] = asn
+            # Persist to DB if possible
+            try:
+                if db_session is not None:
+                    from app.core.models.rdap import RdapCache
+                    row = await db_session.get(RdapCache, norm)
+                    if row is None:
+                        row = RdapCache(ip=norm, asn=asn, source="iptoasn")
+                        db_session.add(row)
+                    else:
+                        row.asn = asn
+                        row.source = "iptoasn"
+                    await db_session.commit()
+            except Exception as e:
+                make_log("ASNResolver", f"DB persist failed for {norm}: {e}", level="warning")
+        return asn
+
+
+resolver = ASNResolver()

app/core/network/config.py

@@ -1,3 +1,4 @@
+import json
 import os
 from typing import List
 
@@ -9,6 +10,23 @@ def _csv_list(val: str) -> List[str]:
     return [x.strip() for x in (val or "").split(",") if x.strip()]
 
 
+def _json_value(val: str, fallback):
+    if not val:
+        return fallback
+    try:
+        return json.loads(val)
+    except Exception:
+        return fallback
+
+
+def _as_list(value):
+    if isinstance(value, list):
+        return value
+    if value is None:
+        return []
+    return [value]
+
+
 # Handshake / network config driven by env
 NODE_PRIVACY = os.getenv("NODE_PRIVACY", NODE_TYPE_PUBLIC).strip().lower()
 if NODE_PRIVACY not in (NODE_TYPE_PUBLIC, NODE_TYPE_PRIVATE):
@@ -30,6 +48,12 @@ NETWORK_TLS_VERIFY = int(os.getenv("NETWORK_TLS_VERIFY", "1")) == 1
 HANDSHAKE_TS_TOLERANCE_SEC = int(os.getenv("HANDSHAKE_TS_TOLERANCE_SEC", "300"))
 HANDSHAKE_RATE_LIMIT_PER_MIN = int(os.getenv("HANDSHAKE_RATE_LIMIT_PER_MIN", "60"))
 
+# IPFS discovery/peering
+IPFS_PRIVATE_BOOTSTRAP_ADDRESSES = _as_list(_json_value(os.getenv("IPFS_PRIVATE_BOOTSTRAP"), []))
+IPFS_PEERING_PEERS = _json_value(os.getenv("IPFS_PEERING_PEERS"), [])
+IPFS_ANNOUNCE_ADDRESSES = _as_list(_json_value(os.getenv("IPFS_ANNOUNCE_ADDRESSES"), []))
+IPFS_NOANNOUNCE_ADDRESSES = _as_list(_json_value(os.getenv("IPFS_NOANNOUNCE_ADDRESSES"), []))
+
 # Capabilities
 NODE_IS_BOOTSTRAP = int(os.getenv("NODE_IS_BOOTSTRAP", "0")) == 1
 MAX_CONTENT_SIZE_MB = int(os.getenv("MAX_CONTENT_SIZE_MB", "512"))

app/core/network/dht/__init__.py

@@ -0,0 +1,35 @@
+"""
+Decentralised storage, replication, and metrics layer.
+"""
+
+from .config import dht_config, DHTConfig
+from .crypto import compute_node_id, compute_content_id, compute_view_id, bits_from_hex, rendezvous_score
+from .keys import MetaKey, MetricKey, MembershipKey
+from .membership import MembershipManager, MembershipState, ReachabilityReceipt
+from .replication import ReplicationManager, ReplicationState, ReplicaLease
+from .metrics import MetricsAggregator, ContentMetricsState, MetricDelta
+from .store import DHTStore
+
+__all__ = [
+    "dht_config",
+    "DHTConfig",
+    "compute_node_id",
+    "compute_content_id",
+    "compute_view_id",
+    "bits_from_hex",
+    "rendezvous_score",
+    "MetaKey",
+    "MetricKey",
+    "MembershipKey",
+    "MembershipManager",
+    "MembershipState",
+    "ReachabilityReceipt",
+    "ReplicationManager",
+    "ReplicationState",
+    "ReplicaLease",
+    "MetricsAggregator",
+    "ContentMetricsState",
+    "MetricDelta",
+    "DHTStore",
+]
+

app/core/network/dht/config.py

@@ -0,0 +1,57 @@
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass
+from functools import lru_cache
+
+
+SCHEMA_VERSION = "v1"
+
+
+def _env_int(name: str, default: int) -> int:
+    try:
+        return int(os.getenv(name, default))
+    except Exception:
+        return default
+
+
+def _env_float(name: str, default: float) -> float:
+    try:
+        return float(os.getenv(name, default))
+    except Exception:
+        return default
+
+
+@dataclass(frozen=True)
+class DHTConfig:
+    """Runtime configuration for the decentralized storage layer."""
+
+    schema_version: str = SCHEMA_VERSION
+    min_receipts: int = _env_int("DHT_MIN_RECEIPTS", 5)
+    min_reachability_ratio: float = _env_float("DHT_MIN_REACHABILITY", 0.6)
+    membership_ttl: int = _env_int("DHT_MEMBERSHIP_TTL", 600)
+    replication_target: int = max(3, _env_int("DHT_REPLICATION_TARGET", 3))
+    lease_ttl: int = _env_int("DHT_LEASE_TTL", 600)
+    heartbeat_interval: int = _env_int("DHT_HEARTBEAT_INTERVAL", 60)
+    heartbeat_miss_threshold: int = _env_int("DHT_HEARTBEAT_MISS_THRESHOLD", 3)
+    rendezvous_base: str = os.getenv("DHT_RENDEZVOUS_HASH", "blake3")
+    pow_difficulty: int = _env_int("DHT_POW_DIFFICULTY", 4)
+    min_asn_diversity: int = _env_int("DHT_MIN_ASN", 3)
+    min_ip_octet_diversity: int = _env_int("DHT_MIN_IP_OCTETS", 3)
+    window_size: int = _env_int("DHT_METRIC_WINDOW_SEC", 3600)
+    default_q: float = _env_float("DHT_MIN_Q", 0.6)
+    seed_refresh_interval: int = _env_int("DHT_SEED_REFRESH_INTERVAL", 30)
+    # Gossip / backoff tuning
+    gossip_interval_sec: int = _env_int("DHT_GOSSIP_INTERVAL_SEC", 30)
+    gossip_backoff_base_sec: int = _env_int("DHT_GOSSIP_BACKOFF_BASE_SEC", 5)
+    gossip_backoff_cap_sec: int = _env_int("DHT_GOSSIP_BACKOFF_CAP_SEC", 600)
+
+
+@lru_cache
+def load_config() -> DHTConfig:
+    """Load configuration with process-wide memoisation."""
+
+    return DHTConfig()
+
+
+dht_config = load_config()

app/core/network/dht/crdt.py

@@ -0,0 +1,278 @@
+from __future__ import annotations
+
+import math
+import time
+from dataclasses import dataclass, field
+from typing import Dict, Any, Iterable, Tuple
+
+from app.core._utils.hash import blake3_hex
+
+
+class CRDTMergeError(RuntimeError):
+    pass
+
+
+class CRDT:
+    def merge(self, other: "CRDT") -> "CRDT":
+        raise NotImplementedError
+
+    def to_dict(self) -> Dict[str, Any]:
+        raise NotImplementedError
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "CRDT":
+        raise NotImplementedError
+
+
+@dataclass
+class LWWElement:
+    value: Any
+    logical_counter: int
+    timestamp: float
+    node_id: str
+
+    def dominates(self, other: "LWWElement") -> bool:
+        if self.logical_counter > other.logical_counter:
+            return True
+        if self.logical_counter < other.logical_counter:
+            return False
+        if self.timestamp > other.timestamp:
+            return True
+        if self.timestamp < other.timestamp:
+            return False
+        # Break all ties by NodeID ordering to guarantee determinism
+        return self.node_id > other.node_id
+
+
+class LWWRegister(CRDT):
+    def __init__(self, element: LWWElement | None = None):
+        self.element = element
+
+    def assign(self, value: Any, logical_counter: int, node_id: str, timestamp: float | None = None) -> None:
+        new_el = LWWElement(value=value, logical_counter=logical_counter, timestamp=timestamp or time.time(), node_id=node_id)
+        if self.element is None or new_el.dominates(self.element):
+            self.element = new_el
+
+    def merge(self, other: "LWWRegister") -> "LWWRegister":
+        if other.element and (self.element is None or other.element.dominates(self.element)):
+            self.element = other.element
+        return self
+
+    def value(self) -> Any:
+        return self.element.value if self.element else None
+
+    def to_dict(self) -> Dict[str, Any]:
+        if not self.element:
+            return {}
+        return {
+            "value": self.element.value,
+            "logical_counter": self.element.logical_counter,
+            "timestamp": self.element.timestamp,
+            "node_id": self.element.node_id,
+        }
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "LWWRegister":
+        if not data:
+            return cls()
+        element = LWWElement(
+            value=data.get("value"),
+            logical_counter=int(data["logical_counter"]),
+            timestamp=float(data["timestamp"]),
+            node_id=str(data["node_id"]),
+        )
+        return cls(element=element)
+
+
+class LWWSet(CRDT):
+    def __init__(self, adds: Dict[str, LWWElement] | None = None, removes: Dict[str, LWWElement] | None = None):
+        self.adds: Dict[str, LWWElement] = adds or {}
+        self.removes: Dict[str, LWWElement] = removes or {}
+
+    def add(self, element_id: str, value: Any, logical_counter: int, node_id: str, timestamp: float | None = None) -> None:
+        elem = LWWElement(value=value, logical_counter=logical_counter, timestamp=timestamp or time.time(), node_id=node_id)
+        existing = self.adds.get(element_id)
+        if not existing or elem.dominates(existing):
+            self.adds[element_id] = elem
+
+    def remove(self, element_id: str, logical_counter: int, node_id: str, timestamp: float | None = None) -> None:
+        elem = LWWElement(value=None, logical_counter=logical_counter, timestamp=timestamp or time.time(), node_id=node_id)
+        existing = self.removes.get(element_id)
+        if not existing or elem.dominates(existing):
+            self.removes[element_id] = elem
+
+    def lookup(self, element_id: str) -> Any | None:
+        add = self.adds.get(element_id)
+        remove = self.removes.get(element_id)
+        if add and (not remove or add.dominates(remove)):
+            return add.value
+        return None
+
+    def elements(self) -> Dict[str, Any]:
+        return {eid: elem.value for eid, elem in self.adds.items() if self.lookup(eid) is not None}
+
+    def merge(self, other: "LWWSet") -> "LWWSet":
+        for eid, elem in other.adds.items():
+            current = self.adds.get(eid)
+            if not current or elem.dominates(current):
+                self.adds[eid] = elem
+        for eid, elem in other.removes.items():
+            current = self.removes.get(eid)
+            if not current or elem.dominates(current):
+                self.removes[eid] = elem
+        return self
+
+    def to_dict(self) -> Dict[str, Any]:
+        def serialize_map(source: Dict[str, LWWElement]) -> Dict[str, Dict[str, Any]]:
+            return {
+                eid: {
+                    "value": elem.value,
+                    "logical_counter": elem.logical_counter,
+                    "timestamp": elem.timestamp,
+                    "node_id": elem.node_id,
+                }
+                for eid, elem in source.items()
+            }
+
+        return {"adds": serialize_map(self.adds), "removes": serialize_map(self.removes)}
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "LWWSet":
+        adds = {
+            eid: LWWElement(
+                value=elem.get("value"),
+                logical_counter=int(elem["logical_counter"]),
+                timestamp=float(elem["timestamp"]),
+                node_id=str(elem["node_id"]),
+            )
+            for eid, elem in (data.get("adds") or {}).items()
+        }
+        removes = {
+            eid: LWWElement(
+                value=elem.get("value"),
+                logical_counter=int(elem["logical_counter"]),
+                timestamp=float(elem["timestamp"]),
+                node_id=str(elem["node_id"]),
+            )
+            for eid, elem in (data.get("removes") or {}).items()
+        }
+        return cls(adds=adds, removes=removes)
+
+
+class PNCounter(CRDT):
+    def __init__(self, increments: Dict[str, int] | None = None, decrements: Dict[str, int] | None = None):
+        self.increments = increments or {}
+        self.decrements = decrements or {}
+
+    def increment(self, node_id: str, value: int = 1) -> None:
+        if value < 0:
+            raise ValueError("value must be non-negative for increment")
+        self.increments[node_id] = self.increments.get(node_id, 0) + value
+
+    def decrement(self, node_id: str, value: int = 1) -> None:
+        if value < 0:
+            raise ValueError("value must be non-negative for decrement")
+        self.decrements[node_id] = self.decrements.get(node_id, 0) + value
+
+    def value(self) -> int:
+        return sum(self.increments.values()) - sum(self.decrements.values())
+
+    def merge(self, other: "PNCounter") -> "PNCounter":
+        for nid, val in other.increments.items():
+            self.increments[nid] = max(self.increments.get(nid, 0), val)
+        for nid, val in other.decrements.items():
+            self.decrements[nid] = max(self.decrements.get(nid, 0), val)
+        return self
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {"inc": dict(self.increments), "dec": dict(self.decrements)}
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "PNCounter":
+        return cls(increments=dict(data.get("inc") or {}), decrements=dict(data.get("dec") or {}))
+
+
+class GCounter(CRDT):
+    def __init__(self, counters: Dict[str, int] | None = None):
+        self.counters = counters or {}
+
+    def increment(self, node_id: str, value: int = 1) -> None:
+        if value < 0:
+            raise ValueError("value must be non-negative")
+        self.counters[node_id] = self.counters.get(node_id, 0) + value
+
+    def value(self) -> int:
+        return sum(self.counters.values())
+
+    def merge(self, other: "GCounter") -> "GCounter":
+        for nid, val in other.counters.items():
+            self.counters[nid] = max(self.counters.get(nid, 0), val)
+        return self
+
+    def to_dict(self) -> Dict[str, Any]:
+        return dict(self.counters)
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "GCounter":
+        return cls(counters=dict(data or {}))
+
+
+def _leading_zeros(value: int, width: int) -> int:
+    if value == 0:
+        return width
+    return width - value.bit_length()
+
+
+@dataclass
+class HyperLogLog(CRDT):
+    precision: int = 12
+    registers: Tuple[int, ...] = field(default_factory=tuple)
+
+    def __post_init__(self) -> None:
+        if not self.registers:
+            self.registers = tuple([0] * (1 << self.precision))
+        else:
+            self.registers = tuple(self.registers)
+
+    @property
+    def m(self) -> int:
+        return len(self.registers)
+
+    def add(self, value: Any) -> None:
+        if value is None:
+            return
+        hashed = int(blake3_hex(str(value).encode()), 16)
+        index = hashed & (self.m - 1)
+        w = hashed >> self.precision
+        rank = _leading_zeros(w, 256 - self.precision) + 1
+        current = self.registers[index]
+        if rank > current:
+            regs = list(self.registers)
+            regs[index] = rank
+            self.registers = tuple(regs)
+
+    def estimate(self) -> float:
+        alpha = 0.7213 / (1 + 1.079 / self.m)
+        indicator = sum(2.0 ** (-r) for r in self.registers)
+        raw = alpha * (self.m ** 2) / indicator
+        if raw <= 2.5 * self.m:
+            zeros = self.registers.count(0)
+            if zeros:
+                return self.m * math.log(self.m / zeros)
+        return raw
+
+    def merge(self, other: "HyperLogLog") -> "HyperLogLog":
+        if self.m != other.m:
+            raise CRDTMergeError("Cannot merge HyperLogLog instances with different precision")
+        merged = [max(a, b) for a, b in zip(self.registers, other.registers)]
+        self.registers = tuple(merged)
+        return self
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {"precision": self.precision, "registers": list(self.registers)}
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "HyperLogLog":
+        if not data:
+            return cls()
+        return cls(precision=int(data.get("precision", 12)), registers=tuple(int(x) for x in data.get("registers", [])))

app/core/network/dht/crypto.py

@@ -0,0 +1,70 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Iterable, Tuple
+
+from app.core._utils.hash import blake3_hex
+
+
+BLAKE3_DIGEST_SIZE = 32
+
+
+def _ensure_bytes(data: Iterable[int] | bytes | bytearray) -> bytes:
+    if isinstance(data, (bytes, bytearray)):
+        return bytes(data)
+    if isinstance(data, str):
+        return data.encode()
+    return bytes(data)
+
+
+def digest_hex(data: Iterable[int] | bytes | bytearray | str) -> str:
+    return blake3_hex(_ensure_bytes(data))
+
+
+def compute_node_id(public_key: bytes) -> str:
+    """NodeID = blake3(pubkey)."""
+
+    if not isinstance(public_key, (bytes, bytearray)):
+        raise TypeError("public_key must be bytes")
+    return digest_hex(public_key)
+
+
+def compute_content_id(encrypted_blob: bytes) -> str:
+    """ContentID = blake3(encrypted_blob)."""
+
+    return digest_hex(encrypted_blob)
+
+
+def compute_view_id(content_id: str, viewer_salt: bytes) -> str:
+    """ViewID = blake3(ContentID||viewer_salt)."""
+
+    if not viewer_salt:
+        raise ValueError("viewer_salt must not be empty")
+    return digest_hex(content_id.encode() + viewer_salt)
+
+
+def bits_from_hex(hex_digest: str, prefix_bits: int) -> Tuple[int, int]:
+    """Extract first prefix_bits from a hex digest. Returns (prefix, total_bits)."""
+
+    if prefix_bits < 0:
+        raise ValueError("prefix_bits must be >= 0")
+    bitstring = bin(int(hex_digest, 16))[2:].zfill(len(hex_digest) * 4)
+    if prefix_bits == 0:
+        return 0, len(bitstring)
+    return int(bitstring[:prefix_bits], 2), len(bitstring)
+
+
+def rendezvous_score(content_id: str, node_id: str) -> int:
+    """Return rendezvous score via blake3(ContentID||NodeID)."""
+    return int(blake3_hex(f"{content_id}:{node_id}".encode()), 16)
+
+
+@dataclass(frozen=True)
+class ContentFingerprint:
+    content_id: str
+    node_id_prefix: int
+    prefix_bits: int
+
+    def matches(self, node_id: str) -> bool:
+        prefix, total = bits_from_hex(node_id, self.prefix_bits)
+        return prefix == self.node_id_prefix and total >= self.prefix_bits

app/core/network/dht/keys.py

@@ -0,0 +1,72 @@
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass
+from datetime import datetime, timezone
+from typing import Dict, Any
+
+from .config import dht_config
+from .crypto import digest_hex
+
+
+def _json_dumps(data: Dict[str, Any]) -> bytes:
+    return json.dumps(data, sort_keys=True, separators=(",", ":")).encode()
+
+
+@dataclass(frozen=True)
+class MetaKey:
+    content_id: str
+    schema_version: str = dht_config.schema_version
+
+    def fingerprint(self) -> str:
+        return digest_hex(self.serialize())
+
+    def serialize(self) -> bytes:
+        return _json_dumps({"schema_version": self.schema_version, "content_id": self.content_id, "type": "meta"})
+
+    def __str__(self) -> str:
+        return f"meta:{self.schema_version}:{self.content_id}"
+
+
+@dataclass(frozen=True)
+class MembershipKey:
+    node_id: str
+    schema_version: str = dht_config.schema_version
+
+    def fingerprint(self) -> str:
+        return digest_hex(self.serialize())
+
+    def serialize(self) -> bytes:
+        return _json_dumps({"schema_version": self.schema_version, "node_id": self.node_id, "type": "membership"})
+
+    def __str__(self) -> str:
+        return f"membership:{self.schema_version}:{self.node_id}"
+
+
+@dataclass(frozen=True)
+class MetricKey:
+    content_id: str
+    window_id: str
+    schema_version: str = dht_config.schema_version
+
+    @classmethod
+    def window_for(cls, timestamp: float, window_size: int | None = None) -> str:
+        win = int(timestamp // (window_size or dht_config.window_size))
+        return datetime.fromtimestamp(win * (window_size or dht_config.window_size), tz=timezone.utc).strftime("%Y%m%d%H")
+
+    def fingerprint(self) -> str:
+        return digest_hex(self.serialize())
+
+    def serialize(self) -> bytes:
+        return _json_dumps(
+            {
+                "schema_version": self.schema_version,
+                "content_id": self.content_id,
+                "window_id": self.window_id,
+                "type": "metric",
+            }
+        )
+
+    def __str__(self) -> str:
+        return f"metric:{self.schema_version}:{self.content_id}:{self.window_id}"
+

app/core/network/dht/membership.py

@@ -0,0 +1,235 @@
+from __future__ import annotations
+
+import ipaddress
+import time
+from dataclasses import dataclass
+from typing import Dict, Any, Iterable, List, Optional, Tuple
+
+from app.core._crypto.signer import Signer
+from .config import dht_config
+from .crdt import LWWSet, HyperLogLog
+from .keys import MembershipKey
+from .store import DHTStore
+
+
+@dataclass
+class ReachabilityReceipt:
+    target_id: str
+    issuer_id: str
+    asn: Optional[int]
+    timestamp: float
+    signature: str
+
+    def as_dict(self) -> Dict[str, Any]:
+        return {
+            "target_id": self.target_id,
+            "issuer_id": self.issuer_id,
+            "asn": self.asn,
+            "timestamp": self.timestamp,
+            "signature": self.signature,
+        }
+
+
+def _ip_first_octet(host: str | None) -> Optional[int]:
+    if not host:
+        return None
+    try:
+        ip = ipaddress.ip_address(host)
+        return int(str(ip).split(".")[0])
+    except Exception:
+        return None
+
+
+class MembershipState:
+    def __init__(self, node_id: str, signer: Signer):
+        self.node_id = node_id
+        self.signer = signer
+        self.members = LWWSet()
+        self.receipts = LWWSet()
+        self.hll = HyperLogLog()
+        self.n_reports: Dict[str, float] = {}
+        self.logical_counter = 0
+
+    def _bump_counter(self) -> int:
+        self.logical_counter += 1
+        return self.logical_counter
+
+    def register_member(
+        self,
+        node_id: str,
+        public_key: str,
+        ip: str | None,
+        asn: Optional[int],
+        metadata: Dict[str, Any] | None = None,
+        timestamp: Optional[float] = None,
+    ) -> None:
+        payload = {
+            "node_id": node_id,
+            "public_key": public_key,
+            "ip": ip,
+            "asn": asn,
+            "ip_first_octet": _ip_first_octet(ip),
+            "meta": metadata or {},
+            "last_update": timestamp or time.time(),
+        }
+        self.members.add(node_id, payload, logical_counter=self._bump_counter(), node_id=self.node_id, timestamp=timestamp)
+        self.hll.add(node_id)
+
+    def forget_member(self, node_id: str) -> None:
+        self.members.remove(node_id, logical_counter=self._bump_counter(), node_id=self.node_id)
+
+    def record_receipt(self, receipt: ReachabilityReceipt) -> None:
+        element_id = f"{receipt.target_id}:{receipt.issuer_id}"
+        self.receipts.add(
+            element_id,
+            receipt.as_dict(),
+            logical_counter=self._bump_counter(),
+            node_id=self.node_id,
+            timestamp=receipt.timestamp,
+        )
+
+    def report_local_population(self) -> None:
+        self.n_reports[self.node_id] = float(self.hll.estimate())
+
+    def merge(self, other: "MembershipState") -> "MembershipState":
+        self.members.merge(other.members)
+        self.receipts.merge(other.receipts)
+        self.hll.merge(other.hll)
+        for node_id, value in other.n_reports.items():
+            self.n_reports[node_id] = max(self.n_reports.get(node_id, 0.0), value)
+        self.logical_counter = max(self.logical_counter, other.logical_counter)
+        return self
+
+    def _unique_asn_for(self, node_id: str) -> Tuple[int, Iterable[int]]:
+        receipts = [
+            entry
+            for rid, entry in self.receipts.elements().items()
+            if entry.get("target_id") == node_id
+        ]
+        unique_asn = {entry.get("asn") for entry in receipts if entry.get("asn") is not None}
+        return len(unique_asn), unique_asn
+
+    def reachability_ratio(self, node_id: str) -> float:
+        unique_count, _ = self._unique_asn_for(node_id)
+        if dht_config.min_receipts <= 0:
+            return 1.0
+        return min(1.0, unique_count / dht_config.min_receipts)
+
+    def active_members(self, include_islands: bool = False) -> List[Dict[str, Any]]:
+        now = time.time()
+        result = []
+        for node_id, data in self.members.elements().items():
+            last_update = data.get("last_update") or 0
+            if now - last_update > dht_config.membership_ttl:
+                continue
+            reachability = self.reachability_ratio(node_id)
+            if not include_islands and reachability < dht_config.default_q:
+                continue
+            enriched = dict(data)
+            enriched["reachability_ratio"] = reachability
+            result.append(enriched)
+        return result
+
+    def n_estimate(self) -> float:
+        self.report_local_population()
+        active_ids = {m["node_id"] for m in self.active_members(include_islands=True)}
+        filtered_reports = [
+            value for node_id, value in self.n_reports.items() if node_id in active_ids and self.reachability_ratio(node_id) >= dht_config.default_q
+        ]
+        local_estimate = float(self.hll.estimate())
+        if filtered_reports:
+            return max(max(filtered_reports), local_estimate)
+        return local_estimate
+
+    def n_estimate_trusted(self, allowed_ids: set[str]) -> float:
+        """Оценка размера сети только по trusted узлам.
+        Берём активных участников, пересекаем с allowed_ids и оцениваем по их числу
+        и по их N_local репортам (если доступны).
+        """
+        self.report_local_population()
+        active_trusted = {m["node_id"] for m in self.active_members(include_islands=True) if m.get("node_id") in allowed_ids}
+        filtered_reports = [
+            value for node_id, value in self.n_reports.items()
+            if node_id in active_trusted and self.reachability_ratio(node_id) >= dht_config.default_q
+        ]
+        # Для доверенных полагаемся на фактическое количество активных Trusted
+        local_estimate = float(len(active_trusted))
+        if filtered_reports:
+            return max(max(filtered_reports), local_estimate)
+        return local_estimate
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {
+            "members": self.members.to_dict(),
+            "receipts": self.receipts.to_dict(),
+            "hll": self.hll.to_dict(),
+            "reports": dict(self.n_reports),
+            "logical_counter": self.logical_counter,
+        }
+
+    @classmethod
+    def from_dict(cls, node_id: str, signer: Signer, data: Dict[str, Any]) -> "MembershipState":
+        inst = cls(node_id=node_id, signer=signer)
+        if data:
+            inst.members = LWWSet.from_dict(data.get("members") or {})
+            inst.receipts = LWWSet.from_dict(data.get("receipts") or {})
+            inst.hll = HyperLogLog.from_dict(data.get("hll") or {})
+            inst.n_reports = {str(k): float(v) for k, v in (data.get("reports") or {}).items()}
+            inst.logical_counter = int(data.get("logical_counter") or 0)
+        return inst
+
+
+class MembershipManager:
+    def __init__(self, node_id: str, signer: Signer, store: DHTStore):
+        self.node_id = node_id
+        self.signer = signer
+        self.store = store
+        self.state = MembershipState(node_id=node_id, signer=signer)
+
+    def _merge_remote(self, data: Dict[str, Any]) -> None:
+        remote_state = MembershipState.from_dict(self.node_id, self.signer, data)
+        self.state.merge(remote_state)
+
+    def ingest_snapshot(self, payload: Dict[str, Any]) -> None:
+        self._merge_remote(payload)
+
+    def register_local(self, public_key: str, ip: str | None, asn: Optional[int], metadata: Dict[str, Any] | None = None) -> None:
+        self.state.register_member(self.node_id, public_key=public_key, ip=ip, asn=asn, metadata=metadata)
+        self._persist()
+
+    def update_member(self, node_id: str, **kwargs) -> None:
+        meta = kwargs.get("metadata") or {}
+        self.state.register_member(
+            node_id,
+            public_key=kwargs.get("public_key", meta.get("public_key")),
+            ip=kwargs.get("ip"),
+            asn=kwargs.get("asn"),
+            metadata=meta,
+        )
+        self._persist()
+
+    def remove_member(self, node_id: str) -> None:
+        self.state.forget_member(node_id)
+        self._persist()
+
+    def record_receipt(self, receipt: ReachabilityReceipt) -> None:
+        self.state.record_receipt(receipt)
+        self._persist()
+
+    def _persist(self) -> None:
+        key = MembershipKey(node_id=self.node_id)
+        self.store.put(
+            key=str(key),
+            fingerprint=key.fingerprint(),
+            value=self.state.to_dict(),
+            logical_counter=self.state.logical_counter,
+            merge_strategy=lambda a, b: MembershipState.from_dict(self.node_id, self.signer, a)
+            .merge(MembershipState.from_dict(self.node_id, self.signer, b))
+            .to_dict(),
+        )
+
+    def n_estimate(self) -> float:
+        return self.state.n_estimate()
+
+    def active_members(self) -> List[Dict[str, Any]]:
+        return self.state.active_members()

app/core/network/dht/metrics.py

@@ -0,0 +1,144 @@
+from __future__ import annotations
+
+import time
+from dataclasses import dataclass
+from typing import Dict, Any, Optional
+
+from app.core._crypto.signer import Signer
+from .config import dht_config
+from .crdt import PNCounter, GCounter, HyperLogLog
+from .crypto import compute_view_id
+from .keys import MetricKey
+from .store import DHTStore
+from .prometheus import update_view_metrics
+
+
+@dataclass
+class MetricDelta:
+    content_id: str
+    view_id: str
+    watch_time: int
+    bytes_out: int
+    completed: bool
+    timestamp: float
+
+    def as_dict(self) -> Dict[str, Any]:
+        return {
+            "content_id": self.content_id,
+            "view_id": self.view_id,
+            "watch_time": self.watch_time,
+            "bytes_out": self.bytes_out,
+            "completed": self.completed,
+            "timestamp": self.timestamp,
+        }
+
+
+class ContentMetricsState:
+    def __init__(self, node_id: str):
+        self.node_id = node_id
+        self.views = PNCounter()
+        self.unique = HyperLogLog()
+        self.watch_time = GCounter()
+        self.bytes_out = GCounter()
+        self.completions = GCounter()
+        self.logical_counter = 0
+
+    def apply(self, delta: MetricDelta) -> None:
+        self.logical_counter += 1
+        self.views.increment(self.node_id, 1)
+        self.unique.add(delta.view_id)
+        if delta.watch_time:
+            self.watch_time.increment(self.node_id, delta.watch_time)
+        if delta.bytes_out:
+            self.bytes_out.increment(self.node_id, delta.bytes_out)
+        if delta.completed:
+            self.completions.increment(self.node_id, 1)
+
+    def merge(self, other: "ContentMetricsState") -> "ContentMetricsState":
+        self.views.merge(other.views)
+        self.unique.merge(other.unique)
+        self.watch_time.merge(other.watch_time)
+        self.bytes_out.merge(other.bytes_out)
+        self.completions.merge(other.completions)
+        self.logical_counter = max(self.logical_counter, other.logical_counter)
+        return self
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {
+            "views": self.views.to_dict(),
+            "unique": self.unique.to_dict(),
+            "watch_time": self.watch_time.to_dict(),
+            "bytes_out": self.bytes_out.to_dict(),
+            "completions": self.completions.to_dict(),
+            "logical_counter": self.logical_counter,
+        }
+
+    @classmethod
+    def from_dict(cls, node_id: str, data: Dict[str, Any]) -> "ContentMetricsState":
+        inst = cls(node_id=node_id)
+        if data:
+            inst.views = PNCounter.from_dict(data.get("views") or {})
+            inst.unique = HyperLogLog.from_dict(data.get("unique") or {})
+            inst.watch_time = GCounter.from_dict(data.get("watch_time") or {})
+            inst.bytes_out = GCounter.from_dict(data.get("bytes_out") or {})
+            inst.completions = GCounter.from_dict(data.get("completions") or {})
+            inst.logical_counter = int(data.get("logical_counter") or 0)
+        return inst
+
+
+class MetricsAggregator:
+    def __init__(self, node_id: str, signer: Signer, store: DHTStore):
+        self.node_id = node_id
+        self.signer = signer
+        self.store = store
+
+    def _load(self, content_id: str, window_id: str) -> ContentMetricsState:
+        key = MetricKey(content_id=content_id, window_id=window_id)
+        record = self.store.get(key.fingerprint())
+        if record:
+            return ContentMetricsState.from_dict(self.node_id, record.value)
+        return ContentMetricsState(node_id=self.node_id)
+
+    def _persist(self, content_id: str, window_id: str, state: ContentMetricsState) -> None:
+        key = MetricKey(content_id=content_id, window_id=window_id)
+        self.store.put(
+            key=str(key),
+            fingerprint=key.fingerprint(),
+            value=state.to_dict(),
+            logical_counter=state.logical_counter,
+            merge_strategy=lambda a, b: ContentMetricsState.from_dict(self.node_id, a)
+            .merge(ContentMetricsState.from_dict(self.node_id, b))
+            .to_dict(),
+        )
+        update_view_metrics(
+            content_id=content_id,
+            window_id=window_id,
+            views=state.views.value(),
+            unique=state.unique.estimate(),
+            watch_time=state.watch_time.value(),
+        )
+
+    def record_view(
+        self,
+        content_id: str,
+        viewer_salt: bytes,
+        watch_time: int,
+        bytes_out: int,
+        completed: bool,
+        timestamp: Optional[float] = None,
+    ) -> MetricDelta:
+        ts = time.time() if timestamp is None else timestamp
+        window_id = MetricKey.window_for(ts)
+        view_id = compute_view_id(content_id, viewer_salt)
+        state = self._load(content_id, window_id)
+        delta = MetricDelta(
+            content_id=content_id,
+            view_id=view_id,
+            watch_time=watch_time,
+            bytes_out=bytes_out,
+            completed=completed,
+            timestamp=ts,
+        )
+        state.apply(delta)
+        self._persist(content_id, window_id, state)
+        return delta