2025-10-26T18:06:05.578758Z :PropagateSessionClosed INFO: Random seed for debugging is 1761501965578705
2025-10-26T18:06:06.109216Z node 1 :METADATA_PROVIDER WARN: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7565593339496398204:8379];send_to=[0:7307199536658146131:7762515];
2025-10-26T18:06:06.109341Z node 1 :METADATA_PROVIDER ERROR: fline=accessor_snapshot_base.cpp:68;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-26T18:06:06.147165Z node 1 :PQ_READ_PROXY DEBUG: Direct read cache created
2025-10-26T18:06:06.166333Z node 2 :METADATA_PROVIDER WARN: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7565593338937621391:4098];send_to=[0:7307199536658146131:7762515];
2025-10-26T18:06:06.167388Z node 2 :METADATA_PROVIDER ERROR: fline=accessor_snapshot_base.cpp:68;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-26T18:06:06.273316Z node 2 :PQ_READ_PROXY DEBUG: Direct read cache created
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/70hy/0002d4/r3tmp/tmp5go2uF/pdisk_1.dat
2025-10-26T18:06:06.652538Z node 1 :IMPORT WARN: Table profiles were not loaded
2025-10-26T18:06:06.702076Z node 1 :HIVE WARN: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-26T18:06:06.702220Z node 1 :HIVE WARN: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-26T18:06:06.704677Z node 1 :HIVE WARN: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-26T18:06:06.704800Z node 1 :HIVE WARN: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-26T18:06:06.707967Z node 1 :HIVE WARN: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-26T18:06:06.709209Z node 1 :HIVE WARN: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-10-26T18:06:06.710574Z node 1 :HIVE WARN: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 4803, node 1
2025-10-26T18:06:06.868023Z node 1 :NET_CLASSIFIER WARN: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/70hy/0002d4/r3tmp/yandexIOhuLo.tmp
2025-10-26T18:06:06.868062Z node 1 :NET_CLASSIFIER WARN: will try to initialize from file: /home/runner/.ya/build/build_root/70hy/0002d4/r3tmp/yandexIOhuLo.tmp
2025-10-26T18:06:06.868385Z node 1 :NET_CLASSIFIER WARN: successfully initialized from file: /home/runner/.ya/build/build_root/70hy/0002d4/r3tmp/yandexIOhuLo.tmp
2025-10-26T18:06:06.868543Z node 1 :NET_CLASSIFIER ERROR: got bad distributable configuration
2025-10-26T18:06:06.939513Z INFO: TTestServer started on Port 8575 GrpcPort 4803
TClient is connected to server localhost:8575
PQClient connected to localhost:4803
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      PlanResolution: 7
      Coo...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-26T18:06:07.239616Z node 1 :FLAT_TX_SCHEMESHARD WARN: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976715657:0, at schemeshard:  72057594046644480
waiting...
waiting...
2025-10-26T18:06:07.307295Z node 1 :FLAT_TX_SCHEMESHARD WARN: NotifyTxCompletion, unknown transaction, txId: 281474976715658, at schemeshard: 72057594046644480
waiting...
waiting...
2025-10-26T18:06:10.003462Z node 1 :KQP_COMPILE_ACTOR ERROR: Compilation failed, self: [1:7565593352381300776:8190], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-26T18:06:10.005090Z node 2 :KQP_COMPILE_ACTOR ERROR: Compilation failed, self: [2:7565593351822523531:4271], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-26T18:06:10.005481Z node 1 :KQP_SESSION WARN: SessionId: ydb://session/3?node_id=1&id=ZTFkMDA0YTQtODI4M2Q4ZTctOTQ5MGVjOGUtYmE0MTNk, ActorId: [1:7565593352381300774:8382], ActorState: ExecuteState, TraceId: 01k8gv5zbb5p3q1qgqjd4zardt, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-26T18:06:10.007816Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-26T18:06:10.009647Z node 1 :FLAT_TX_SCHEMESHARD WARN: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715661:0, at schemeshard:  72057594046644480
2025-10-26T18:06:10.009877Z node 2 :KQP_SESSION WARN: SessionId: ydb://session/3?node_id=2&id=ODU2OTM2OS04MTYyZGMyNS04NWY0YzVjMC04YjM0ZWVmZQ==, ActorId: [2:7565593351822523528:4270], ActorState: ExecuteState, TraceId: 01k8gv5zgffrwej7ecjxyk9gez, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-26T18:06:10.015093Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-26T18:06:10.169887Z node 1 :FLAT_TX_SCHEMESHARD WARN: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715662:0, at schemeshard:  72057594046644480
2025-10-26T18:06:10.294051Z node 1 :FLAT_TX_SCHEMESHARD WARN: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715663:0, at schemeshard:  72057594046644480
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:4803", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-26T18:06:10.618286Z node 1 :KQP_EXECUTER ERROR: TxId: 281474976715664. Ctx: { TraceId: 01k8gv5zze404vf6k6eqgrt6gv, Database: , DatabaseId: /Root, SessionId: ydb://session/3?node_id=1&id=NjY2NGU0MzQtZTlmNjc2YTEtYjVjZDQ0OWMtNjZjODE0OTU=, CurrentExecutionId: , CustomerSuppliedId: , PoolId: }. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7565593356676268466:12334] 
2025-10-26T18:06:11.107633Z node 1 :METADATA_PROVIDER ERROR: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7565593339496398204:8379];send_to=[0:7307199536658146131:7762515];
2025-10-26T18:06:11.107766Z node 1 :METADATA_PROVIDER ERROR: fline=accessor_snapshot_base.cpp:68;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-10-26T18:06:11.166943Z node 2 :METADATA_PROVIDER ERROR: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7565593338937621391:4098];send_to=[0:7307199536658146131:7762515];
2025-10-26T18:06:11.167039Z node 2 :METADATA_PROVIDER ERROR: fline=accessor_snapshot_base.cpp:68;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-26T18:06:16.517041Z node 1 :FLAT_TX_SCHEMESHARD WARN: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715676:0, at schemeshard:  72057594046644480
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:4803
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-26T18:06:16.621125Z node 1 :PERSQUEUE INFO: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:4803
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      ReadRules: "user"
      ReadFromTimestampsMs: 0
      ConsumerFormatVersions: 0
      ConsumerCodecs {
      }
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      ReadRuleVersions: 0
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976715678
  SchemeShardTabletId: 72057594046644480
  PathId: 9
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-26T18:06:16.678046Z node 1 :PERSQUEUE_READ_BALANCER INFO: balancer: [72075186224037893] topic  pipe [1:7565593382446072844:8298] connected; active server actors: 1
2025-10-26T18:06:16.678309Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: balancer: [72075186224037893] topic rt3.dc1--test-topic updating configuration. Deleted partitions []. Added partitions [0]
2025-10-26T18:06:16.679015Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-26T18:06:16.680524Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-26T18:06:16.681188Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: tablet 72075186224037893 topic rt3.dc1--test-topic TEvClientConnected TabletId 72057594046644480, NodeId 1, Generation 2
2025-10-26T18:06:16.693045Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-26T18:06:16.693469Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Registered with mediator time cast
2025-10-26T18:06:16.696772Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-26T18:06:16.697060Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] doesn't have tx info
2025-10-26T18:06:16.697107Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] PlanStep 0, PlanTxId 0, ExecStep 0, ExecTxId 0
2025-10-26T18:06:16.697125Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-26T18:06:16.697157Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Txs.size=0, PlannedTxs.size=0
2025-10-26T18:06:16.697183Z node 2 :PERSQUEUE NOTICE: Tablet 72075186224037892 disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-26T18:06:16.697232Z node 2 :PERSQUEUE INFO: Tablet 72075186224037892 doesn't have tx writes info
2025-10-26T18:06:16.698823Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 server connected, pipe [1:7565593382446072879:7], now have 1 active actors on pipe
2025-10-26T18:06:16.698948Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: tablet 72075186224037893 topic rt3.dc1--test-topic TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-10-26T18:06:16.718288Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 server connected, pipe [1:7565593382446072842:8367], now have 1 active actors on pipe
2025-10-26T18:06:16.718915Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Handle TEvPersQueue::TEvProposeTransaction SourceActor { RawX1: 7565593339496398301 RawX2: 4294979628 } TxId: 281474976715678 Config { TabletConfig { PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true ReadRules: "user" ReadFromTimestampsMs: 0 ConsumerFormatVersions: 0 ConsumerCodecs { } Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } ReadRuleVersions: 0 TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } } }
2025-10-26T18:06:16.719054Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Propose TxId 281474976715678, WriteId (empty maybe)
2025-10-26T18:06:16.719106Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Try execute txs with state UNKNOWN
2025-10-26T18:06:16.719126Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, State UNKNOWN
2025-10-26T18:06:16.719152Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] schedule TEvProposeTransactionResult(PREPARED)
2025-10-26T18:06:16.719173Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, NewState PREPARING
2025-10-26T18:06:16.719245Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] write key for TxId 281474976715678
2025-10-26T18:06:16.719553Z node 2 :PERSQUEUE DEBUG: [TxId: 281474976715678] save tx TxId: 281474976715678 State: PREPARED MinStep: 1761501976626 MaxStep: 18446744073709551615 Kind: KIND_CONFIG TabletConfig { PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true ReadRules: "user" ReadFromTimestampsMs: 0 ConsumerFormatVersions: 0 ConsumerCodecs { } Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } ReadRuleVersions: 0 TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } ReadRuleGenerations: 0 Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Codec { } Version: 0 Generation: 0 Important: false } } BootstrapConfig { } SourceActor { RawX1: 7565593339496398301 RawX2: 4294979628 }
2025-10-26T18:06:16.719673Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-26T18:06:16.724485Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-26T18:06:16.724548Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Try execute txs with state PREPARING
2025-10-26T18:06:16.724565Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, State PREPARING
2025-10-26T18:06:16.724584Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, NewState PREPARED
2025-10-26T18:06:16.733852Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Handle TEvPersQueue::TEvProposeTransactionAttach TabletId: 72075186224037892 TxId: 281474976715678
2025-10-26T18:06:16.733920Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 server connected, pipe [1:7565593382446072912:8566], now have 1 active actors on pipe
2025-10-26T18:06:16.741542Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Handle TEvTxProcessing::TEvPlanStep Transactions { TxId: 281474976715678 AckTo { RawX1: 7565593339496398552 RawX2: 4294975574 } } Step: 1761501976773 MediatorID: 72057594046382081 TabletID: 72075186224037892
2025-10-26T18:06:16.741634Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Try execute txs with state PREPARED
2025-10-26T18:06:16.741660Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, State PREPARED
2025-10-26T18:06:16.741705Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, NewState PLANNING
2025-10-26T18:06:16.741735Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] PlanStep 1761501976773, PlanTxId 281474976715678
2025-10-26T18:06:16.741783Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] write key for TxId 281474976715678
2025-10-26T18:06:16.742177Z node 2 :PERSQUEUE DEBUG: [TxId: 281474976715678] save tx TxId: 281474976715678 State: PLANNED MinStep: 1761501976626 MaxStep: 18446744073709551615 Step: 1761501976773 Kind: KIND_CONFIG TabletConfig { PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true ReadRules: "user" ReadFromTimestampsMs: 0 ConsumerFormatVersions: 0 ConsumerCodecs { } Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } ReadRuleVersions: 0 TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } ReadRuleGenerations: 0 Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Codec { } Version: 0 Generation: 0 Important: false } } BootstrapConfig { } SourceActor { RawX1: 7565593339496398301 RawX2: 4294979628 }
2025-10-26T18:06:16.742312Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-26T18:06:16.753404Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-26T18:06:16.753464Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Try execute txs with state PLANNING
2025-10-26T18:06:16.753480Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, State PLANNING
2025-10-26T18:06:16.753504Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, NewState PLANNED
2025-10-26T18:06:16.753535Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678 moved from PLANNING to PLANNED
2025-10-26T18:06:16.753550Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxQueue.size 1
2025-10-26T18:06:16.753569Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] New ExecStep 1761501976773, ExecTxId 281474976715678
2025-10-26T18:06:16.754517Z node 2 :PERSQUEUE DEBUG: Initializing topic 'rt3.dc1--test-topic' partition 0. Step TInitConfigStep
2025-10-26T18:06:16.754582Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, NewState CALCULATING
2025-10-26T18:06:16.754608Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678 moved from PLANNED to CALCULATING
2025-10-26T18:06:16.754614Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-26T18:06:16.754984Z node 2 :PERSQUEUE DEBUG: Initializing topic 'rt3.dc1--test-topic' partition 0. Step TInitInternalFieldsStep
2025-10-26T18:06:16.755409Z node 2 :PERSQUEUE INFO: bootstrapping 0 [2:7565593381887295248:4296]
2025-10-26T18:06:16.758663Z node 2 :PERSQUEUE DEBUG: Initializing topic 'rt3.dc1--test-topic' partition 0. Completed.
2025-10-26T18:06:16.758838Z node 2 :PERSQUEUE INFO: init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [2:7565593381887295248:4296]
2025-10-26T18:06:16.758874Z node 2 :PERSQUEUE DEBUG: SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-26T18:06:16.760429Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] Process pending events. Count 1
2025-10-26T18:06:16.760647Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-26T18:06:16.760985Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Handle TEvPQ::TEvProposePartitionConfigResult Step 1761501976773, TxId 281474976715678, Partition 0
2025-10-26T18:06:16.761007Z node 2 :PERSQUEUE DEBUG: [TxId: 281474976715678] Handle TEvProposePartitionConfigResult
2025-10-26T18:06:16.761026Z node 2 :PERSQUEUE DEBUG: [TxId: 281474976715678] Partition responses 1/1
2025-10-26T18:06:16.761059Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Try execute txs with state CALCULATING
2025-10-26T18:06:16.761084Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, State CALCULATING
2025-10-26T18:06:16.761106Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678 State CALCULATING FrontTxId 281474976715678
2025-10-26T18:06:16.761127Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Received 1, Expected 1
2025-10-26T18:06:16.761159Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, NewState CALCULATED
2025-10-26T18:06:16.761187Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678 moved from CALCULATING to CALCULATED
2025-10-26T18:06:16.761216Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] write key for TxId 281474976715678
2025-10-26T18:06:16.761652Z node 2 :PERSQUEUE DEBUG: [TxId: 281474976715678] save tx TxId: 281474976715678 State: CALCULATED MinStep: 1761501976626 MaxStep: 18446744073709551615 Step: 1761501976773 Predicate: true Kind: KIND_CONFIG TabletConfig { PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true ReadRules: "user" ReadFromTimestampsMs: 0 ConsumerFormatVersions: 0 ConsumerCodecs { } Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } ReadRuleVersions: 0 TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } ReadRuleGenerations: 0 Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Codec { } Version: 0 Generation: 0 Important: false } } BootstrapConfig { } SourceActor { RawX1: 7565593339496398301 RawX2: 4294979628 }
2025-10-26T18:06:16.761825Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-26T18:06:16.777635Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-26T18:06:16.777674Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Try execute txs with state CALCULATED
2025-10-26T18:06:16.777704Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, State CALCULATED
2025-10-26T18:06:16.777734Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678 State CALCULATED FrontTxId 281474976715678
2025-10-26T18:06:16.777761Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, NewState WAIT_RS
2025-10-26T18:06:16.777788Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678 moved from CALCULATED to WAIT_RS
2025-10-26T18:06:16.777819Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Send TEvTxProcessing::TEvReadSet to 0 receivers. Wait TEvTxProcessing::TEvReadSet from 0 senders.
2025-10-26T18:06:16.777846Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] HaveParticipantsDecision 1
2025-10-26T18:06:16.777882Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, NewState EXECUTING
2025-10-26T18:06:16.777904Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678 moved from WAIT_RS to EXECUTING
2025-10-26T18:06:16.777921Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Received 0, Expected 1
2025-10-26T18:06:16.778007Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] Handle TEvPQ::TEvTxCommit Step 1761501976773, TxId 281474976715678
2025-10-26T18:06:16.778764Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:16.782656Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:16.782792Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Handle TEvPQ::TEvTxCommitDone Step 1761501976773, TxId 281474976715678, Partition 0
2025-10-26T18:06:16.782818Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Try execute txs with state EXECUTING
2025-10-26T18:06:16.782838Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, State EXECUTING
2025-10-26T18:06:16.782858Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678 State EXECUTING FrontTxId 281474976715678
2025-10-26T18:06:16.782878Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Received 1, Expected 1
2025-10-26T18:06:16.782901Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId: 281474976715678 send TEvPersQueue::TEvProposeTransactionResult(COMPLETE)
2025-10-26T18:06:16.782939Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] complete TxId 281474976715678
2025-10-26T18:06:16.783283Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true ReadRules: "user" ReadFromTimestampsMs: 0 ConsumerFormatVersions: 0 ConsumerCodecs { } Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } ReadRuleVersions: 0 TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } ReadRuleGenerations: 0 Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Codec { } Version: 0 Generation: 0 Important: false }
2025-10-26T18:06:16.783384Z node 2 :PERSQUEUE NOTICE: Tablet 72075186224037892 disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-26T18:06:16.783449Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] delete partitions for TxId 281474976715678
2025-10-26T18:06:16.783471Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, NewState EXECUTED
2025-10-26T18:06:16.783517Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678 moved from EXECUTING to EXECUTED
2025-10-26T18:06:16.783544Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] write key for TxId 281474976715678
2025-10-26T18:06:16.783873Z node 2 :PERSQUEUE DEBUG: [TxId: 281474976715678] save tx TxId: 281474976715678 State: EXECUTED MinStep: 1761501976626 MaxStep: 18446744073709551615 Step: 1761501976773 Predicate: true Kind: KIND_CONFIG TabletConfig { PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true ReadRules: "user" ReadFromTimestampsMs: 0 ConsumerFormatVersions: 0 ConsumerCodecs { } Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } ReadRuleVersions: 0 TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } ReadRuleGenerations: 0 Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Codec { } Version: 0 Generation: 0 Important: false } } BootstrapConfig { } SourceActor { RawX1: 7565593339496398301 RawX2: 4294979628 }
2025-10-26T18:06:16.784073Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-26T18:06:16.787825Z node 1 :KQP_EXECUTER ERROR: TxId: 281474976715680. Ctx: { TraceId: 01k8gv6631amj3fh8ht5ydgy07, Database: , DatabaseId: /Root, SessionId: ydb://session/3?node_id=1&id=ZDI1MDBlZDAtNTBlODI1OS0xZjljOTdhZi1kYzJiN2Q1MQ==, CurrentExecutionId: , CustomerSuppliedId: , PoolId: }. Database not set, use /Root
2025-10-26T18:06:16.791892Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-26T18:06:16.791932Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Try execute txs with state EXECUTED
2025-10-26T18:06:16.791946Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, State EXECUTED
2025-10-26T18:06:16.791963Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678 State EXECUTED FrontTxId 281474976715678
2025-10-26T18:06:16.791984Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TPersQueue::SendEvReadSetAckToSenders
2025-10-26T18:06:16.792010Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, NewState WAIT_RS_ACKS
2025-10-26T18:06:16.792037Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678 moved from EXECUTED to WAIT_RS_ACKS
2025-10-26T18:06:16.792062Z node 2 :PERSQUEUE DEBUG: [TxId: 281474976715678] PredicateAcks: 0/0
2025-10-26T18:06:16.792072Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] HaveAllRecipientsReceive 1, AllSupportivePartitionsHaveBeenDeleted 1
2025-10-26T18:06:16.792089Z node 2 :PERSQUEUE DEBUG: [TxId: 281474976715678] PredicateAcks: 0/0
2025-10-26T18:06:16.792104Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] add an TxId 281474976715678 to the list for deletion
2025-10-26T18:06:16.792129Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, NewState DELETING
2025-10-26T18:06:16.792165Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] delete key for TxId 281474976715678
2025-10-26T18:06:16.792231Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-26T18:06:16.795242Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-26T18:06:16.795276Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] Try execute txs with state DELETING
2025-10-26T18:06:16.795291Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] TxId 281474976715678, State DELETING
2025-10-26T18:06:16.795315Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892] delete TxId 281474976715678
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 9
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976715678
    CreateStep: 1761501976773
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    BalancerTabletID: 72075186224037893
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 9
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        Sour...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 9
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976715678
    CreateStep: 1761501976773
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    BalancerTabletID: 72075186224037893
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 9
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      ReadRules: "user"
      ReadFromTimestampsMs: 0
      ConsumerFormatVersions: 0
      ConsumerCodecs {
      }
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      ReadRuleVersions: 0
      YdbDatabasePath: "/Root"
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 8
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:4803
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-26T18:06:16.833553Z node 1 :PERSQUEUE INFO: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:4803
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
2025-10-26T18:06:17.342195Z node 1 :PERSQUEUE INFO: proxy answer <none>
CallPersQueueGRPC request to localhost:4803
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        ReadRules: "user"
        ReadFromTimestampsMs: 0
        ConsumerFormatVersions: 0
        ConsumerCodecs {
        }
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        ReadRuleVersions: 0
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Codec {
          }
          Version: 0
...
(TRUNCATED)
=== Topic created, have version: 1
2025-10-26T18:06:17.848809Z node 1 :PERSQUEUE INFO: proxy answer <none>
2025-10-26T18:06:17.863176Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-26T18:06:17.864681Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-26T18:06:17.864720Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:4803
2025-10-26T18:06:17.889795Z node 1 :PQ_WRITE_PROXY DEBUG: new grpc connection
2025-10-26T18:06:17.889870Z node 1 :PQ_WRITE_PROXY DEBUG: new session created cookie 1
2025-10-26T18:06:17.901449Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-26T18:06:17.905921Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-26T18:06:17.906109Z node 1 :PQ_WRITE_PROXY INFO: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:50318
2025-10-26T18:06:17.906129Z node 1 :PQ_WRITE_PROXY INFO: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:50318 proto=v1  topic=test-topic durationSec=0
2025-10-26T18:06:17.906142Z node 1 :PQ_WRITE_PROXY INFO: init check schema
2025-10-26T18:06:17.908067Z node 1 :PQ_WRITE_PROXY INFO: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-26T18:06:17.908261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-26T18:06:17.908272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-26T18:06:17.908284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-26T18:06:17.908314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593386741040405:8390] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-26T18:06:17.910360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593386741040405:8390] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-26T18:06:18.084922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593386741040405:8390] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-26T18:06:18.085370Z node 1 :PERSQUEUE_READ_BALANCER INFO: balancer: [72075186224037893] topic rt3.dc1--test-topic pipe [1:7565593391036007743:8390] connected; active server actors: 1
2025-10-26T18:06:18.085565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593386741040405:8390] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-26T18:06:18.085609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593386741040405:8390] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-26T18:06:18.086318Z node 1 :PERSQUEUE_READ_BALANCER INFO: balancer: [72075186224037893] topic rt3.dc1--test-topic pipe [1:7565593391036007743:8390] disconnected; active server actors: 1
2025-10-26T18:06:18.086340Z node 1 :PERSQUEUE_READ_BALANCER INFO: balancer: [72075186224037893] topic rt3.dc1--test-topic pipe [1:7565593391036007743:8390] disconnected no session
2025-10-26T18:06:18.199578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593386741040405:8390] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-26T18:06:18.199636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593386741040405:8390] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-26T18:06:18.199676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593386741040405:8390] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-26T18:06:18.199708Z node 1 :PQ_WRITE_PROXY DEBUG: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-26T18:06:18.202750Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-26T18:06:18.204852Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 server connected, pipe [1:7565593391036007769:8390], now have 1 active actors on pipe
2025-10-26T18:06:18.204937Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:18.204961Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:18.205092Z node 2 :PERSQUEUE INFO: new Cookie src|66d5ed-86806240-9acf7c31-6a968dee_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-26T18:06:18.205181Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyOwnerOk. Partition: 0
2025-10-26T18:06:18.205250Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-26T18:06:18.209968Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:18.210001Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:18.210146Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-26T18:06:18.210785Z node 1 :PQ_WRITE_PROXY INFO: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|66d5ed-86806240-9acf7c31-6a968dee_0
2025-10-26T18:06:18.214111Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1761501978214 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-26T18:06:18.214281Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|66d5ed-86806240-9acf7c31-6a968dee_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-26T18:06:18.214543Z :INFO: [] MessageGroupId [src] SessionId [src|66d5ed-86806240-9acf7c31-6a968dee_0] Write session: close. Timeout = 0 ms
2025-10-26T18:06:18.214574Z :INFO: [] MessageGroupId [src] SessionId [src|66d5ed-86806240-9acf7c31-6a968dee_0] Write session will now close
2025-10-26T18:06:18.214611Z :DEBUG: [] MessageGroupId [src] SessionId [src|66d5ed-86806240-9acf7c31-6a968dee_0] Write session: aborting
2025-10-26T18:06:18.215016Z :INFO: [] MessageGroupId [src] SessionId [src|66d5ed-86806240-9acf7c31-6a968dee_0] Write session: gracefully shut down, all writes complete
2025-10-26T18:06:18.215050Z :DEBUG: [] MessageGroupId [src] SessionId [src|66d5ed-86806240-9acf7c31-6a968dee_0] Write session: destroy
2025-10-26T18:06:18.218784Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 1 sessionId: src|66d5ed-86806240-9acf7c31-6a968dee_0 grpc read done: success: 0 data: 
2025-10-26T18:06:18.218815Z node 1 :PQ_WRITE_PROXY INFO: session v1 cookie: 1 sessionId: src|66d5ed-86806240-9acf7c31-6a968dee_0 grpc read failed
2025-10-26T18:06:18.218849Z node 1 :PQ_WRITE_PROXY INFO: session v1 cookie: 1 sessionId: src|66d5ed-86806240-9acf7c31-6a968dee_0 grpc closed
2025-10-26T18:06:18.218866Z node 1 :PQ_WRITE_PROXY INFO: session v1 cookie: 1 sessionId: src|66d5ed-86806240-9acf7c31-6a968dee_0 is DEAD
2025-10-26T18:06:18.219621Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-26T18:06:18.220402Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 server disconnected, pipe [1:7565593391036007769:8390] destroyed
2025-10-26T18:06:18.220487Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::DropOwner.
Session was created
>>> Ready to answer: ok
2025-10-26T18:06:18.311933Z :INFO: [/Root] [] [27b2560e-b1b66188-ad524d99-b60aea45] Open read subsessions to databases: { name: dc1, endpoint: localhost:4803, path: /Root }, { name: dc2, endpoint: localhost:4803, path: /Invalid }
2025-10-26T18:06:18.327610Z :INFO: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] Starting read session
2025-10-26T18:06:18.327664Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] Starting single session
2025-10-26T18:06:18.327746Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [] Write session: try to update token
2025-10-26T18:06:18.340501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Write session: Do CDS request
2025-10-26T18:06:18.342073Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Start write session. Will connect to endpoint: localhost:4803
2025-10-26T18:06:18.346953Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-26T18:06:18.347016Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-26T18:06:18.347052Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Reconnecting session to cluster  in 0.000000s
2025-10-26T18:06:18.354543Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Successfully connected. Initializing session
2025-10-26T18:06:18.365226Z :INFO: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Starting read session
2025-10-26T18:06:18.365268Z :DEBUG: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Starting single session
2025-10-26T18:06:18.365383Z :DEBUG: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-26T18:06:18.365417Z :DEBUG: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-26T18:06:18.365448Z :DEBUG: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] [] Reconnecting session to cluster  in 0.000000s
2025-10-26T18:06:18.365614Z :ERROR: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] [] Got error. Status: NOT_FOUND. Description: <main>: Error: Database Invalid does not exist, code: 200202 <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. 
2025-10-26T18:06:18.365713Z :DEBUG: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-26T18:06:18.365814Z :DEBUG: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-26T18:06:18.365974Z :INFO: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] [] Closing session to cluster: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:18.365783Z node 1 :PQ_READ_PROXY DEBUG: new grpc connection
2025-10-26T18:06:18.365812Z node 1 :PQ_READ_PROXY DEBUG: new session created cookie 1
2025-10-26T18:06:18.370446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-26T18:06:18.377257Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-26T18:06:18.377574Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 read init: from# ipv6:[::1]:50340, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-26T18:06:18.380830Z node 1 :PQ_WRITE_PROXY DEBUG: new grpc connection
2025-10-26T18:06:18.380861Z node 1 :PQ_WRITE_PROXY DEBUG: new session created cookie 2
2025-10-26T18:06:18.381467Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-26T18:06:18.381589Z node 1 :PQ_WRITE_PROXY INFO: session request cookie: 2 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:50340
2025-10-26T18:06:18.381604Z node 1 :PQ_WRITE_PROXY INFO: write session:  cookie=2 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:50340 proto=v1  topic=test-topic durationSec=0
2025-10-26T18:06:18.381616Z node 1 :PQ_WRITE_PROXY INFO: init check schema
2025-10-26T18:06:18.381710Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 auth for : user
2025-10-26T18:06:18.385565Z :INFO: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Server session id: shared/user_1_1_1719487083443976093_v1
2025-10-26T18:06:18.385609Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] In ContinueReadingDataImpl, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-26T18:06:18.383336Z node 1 :PQ_WRITE_PROXY INFO: session v1 cookie: 2 sessionId:  describe result for acl check
2025-10-26T18:06:18.385857Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-26T18:06:18.383463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-26T18:06:18.383473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-26T18:06:18.383503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-26T18:06:18.383520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593391036007794:8439] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-26T18:06:18.383568Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Handle describe topics response
2025-10-26T18:06:18.383709Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 auth is DEAD
2025-10-26T18:06:18.383768Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 auth ok: topics# 1, initDone# 0
2025-10-26T18:06:18.384950Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 register session: topic# rt3.dc1--test-topic
2025-10-26T18:06:18.386113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593391036007794:8439] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-26T18:06:18.386301Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 1, data# { read_request { bytes_size: 524288 } }
2025-10-26T18:06:18.386430Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 got read request: guid# 443e71fe-5d719496-4aa9fcb6-6d8ecf86
2025-10-26T18:06:18.386548Z node 1 :PERSQUEUE_READ_BALANCER INFO: balancer: [72075186224037893] topic rt3.dc1--test-topic pipe [1:7565593391036007795:8438] connected; active server actors: 1
2025-10-26T18:06:18.386588Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: balancer: [72075186224037893] topic rt3.dc1--test-topic consumer "user" register session for pipe [1:7565593391036007795:8438] session shared/user_1_1_1719487083443976093_v1
2025-10-26T18:06:18.386651Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: balancer: [72075186224037893] topic rt3.dc1--test-topic consumer user register readable partition 0
2025-10-26T18:06:18.386730Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: balancer: [72075186224037893] topic rt3.dc1--test-topic consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-26T18:06:18.386781Z node 1 :PERSQUEUE_READ_BALANCER INFO: balancer: [72075186224037893] topic rt3.dc1--test-topic consumer user register reading session ReadingSession "shared/user_1_1_1719487083443976093_v1" (Sender=[1:7565593391036007789:8438], Pipe=[1:7565593391036007795:8438], Partitions=[], ActiveFamilyCount=0)
2025-10-26T18:06:18.386816Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: balancer: [72075186224037893] topic rt3.dc1--test-topic consumer user rebalancing was scheduled
2025-10-26T18:06:18.386866Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: balancer: [72075186224037893] topic rt3.dc1--test-topic consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-26T18:06:18.386924Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: balancer: [72075186224037893] topic rt3.dc1--test-topic consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_1_1719487083443976093_v1" (Sender=[1:7565593391036007789:8438], Pipe=[1:7565593391036007795:8438], Partitions=[], ActiveFamilyCount=0)
2025-10-26T18:06:18.386984Z node 1 :PERSQUEUE_READ_BALANCER INFO: balancer: [72075186224037893] topic rt3.dc1--test-topic consumer user family 1 status Active partitions [0] session "shared/user_1_1_1719487083443976093_v1" sender [1:7565593391036007789:8438] lock partition 0 for ReadingSession "shared/user_1_1_1719487083443976093_v1" (Sender=[1:7565593391036007789:8438], Pipe=[1:7565593391036007795:8438], Partitions=[], ActiveFamilyCount=1) generation 1 step 1
2025-10-26T18:06:18.387067Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: balancer: [72075186224037893] topic rt3.dc1--test-topic consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-26T18:06:18.387110Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: balancer: [72075186224037893] topic rt3.dc1--test-topic consumer user balancing duration: 0.000210s
2025-10-26T18:06:18.388009Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 1 Session: "shared/user_1_1_1719487083443976093_v1" ClientId: "user" PipeClient { RawX1: 7565593391036007795 RawX2: 4503603922346230 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-26T18:06:18.388120Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-26T18:06:18.389010Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1
2025-10-26T18:06:18.388700Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 server connected, pipe [1:7565593391036007800:8440], now have 1 active actors on pipe
2025-10-26T18:06:18.389615Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:18.389637Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:18.389678Z node 2 :PERSQUEUE DEBUG: Created session shared/user_1_1_1719487083443976093_v1 on pipe: [1:7565593391036007800:8440]
2025-10-26T18:06:18.393614Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 0 WriteTimestampMS: 1761501976758 CreateTimestampMS: 1761501976758 SizeLag: 0 WriteTimestampEstimateMS: 1761501978381 } Cookie: 18446744073709551615 }
2025-10-26T18:06:18.389868Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_1_1719487083443976093_v1
2025-10-26T18:06:18.393670Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 0 readOffset 0 committedOffset 0
2025-10-26T18:06:18.390041Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:18.393781Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 sending to client partition status
2025-10-26T18:06:18.390532Z node 2 :PQ_READ_PROXY DEBUG: Direct read cache: registered server session: shared/user_1_1_1719487083443976093_v1:1 with generation 1
2025-10-26T18:06:18.393066Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:18.393134Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-26T18:06:18.397657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593391036007794:8439] (SourceId=src_id, PreferedPartition=(NULL)) RequestPQRB
2025-10-26T18:06:18.398144Z node 1 :PERSQUEUE_READ_BALANCER INFO: balancer: [72075186224037893] topic rt3.dc1--test-topic pipe [1:7565593391036007817:8439] connected; active server actors: 1
2025-10-26T18:06:18.398296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593391036007794:8439] (SourceId=src_id, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src_id
2025-10-26T18:06:18.398321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593391036007794:8439] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-26T18:06:18.398843Z node 1 :PERSQUEUE_READ_BALANCER INFO: balancer: [72075186224037893] topic rt3.dc1--test-topic pipe [1:7565593391036007817:8439] disconnected; active server actors: 1
2025-10-26T18:06:18.398867Z node 1 :PERSQUEUE_READ_BALANCER INFO: balancer: [72075186224037893] topic rt3.dc1--test-topic pipe [1:7565593391036007817:8439] disconnected no session
2025-10-26T18:06:18.407486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593391036007794:8439] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-26T18:06:18.407534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593391036007794:8439] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-26T18:06:18.407558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593391036007794:8439] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-26T18:06:18.407585Z node 1 :PQ_WRITE_PROXY DEBUG: ProceedPartition. session cookie: 2 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-26T18:06:18.408704Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-26T18:06:18.411395Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 server connected, pipe [1:7565593391036007828:8439], now have 1 active actors on pipe
2025-10-26T18:06:18.411465Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:18.411484Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:18.411572Z node 2 :PERSQUEUE INFO: new Cookie src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-26T18:06:18.411661Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyOwnerOk. Partition: 0
2025-10-26T18:06:18.411717Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-26T18:06:18.417042Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:18.417088Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:18.417192Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-26T18:06:18.418654Z node 1 :PQ_WRITE_PROXY INFO: session inited cookie: 2 partition: 0 MaxSeqNo: 0 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0
2025-10-26T18:06:18.419538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1761501978419 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-26T18:06:18.419700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Write session established. Init response: session_id: "src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-26T18:06:18.422427Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 1 to 1
2025-10-26T18:06:18.424581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:18.424626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 1
>>> Got event: StartPartitionSession { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 CommittedOffset: 0 EndOffset: 0 }
2025-10-26T18:06:18.435320Z :INFO: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Confirm partition stream create. Partition stream id: 1. Cluster: "-". Topic: "test-topic". Partition: 0. Read offset: (empty maybe)
2025-10-26T18:06:18.454225Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 2 to 2
2025-10-26T18:06:18.456161Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:18.456207Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 2
2025-10-26T18:06:18.461887Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 1, data# { start_partition_session_response { partition_session_id: 1 } }
2025-10-26T18:06:18.462052Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 got StartRead from client: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 0, commitOffset# (empty maybe)
2025-10-26T18:06:18.462128Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Start reading TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 0 readOffset 0 committedOffset 0 clientCommitOffset (empty maybe) clientReadOffset 0
2025-10-26T18:06:18.462176Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 1 from offset0
2025-10-26T18:06:18.482044Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-26T18:06:18.482095Z :DEBUG: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] [] Abort session to cluster
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:18.497528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 3 to 3
2025-10-26T18:06:18.508128Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:18.508183Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 3
2025-10-26T18:06:18.513565Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:18.529797Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:18.539305Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 4 to 4
2025-10-26T18:06:18.539317Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:18.542111Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:18.542157Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 4
2025-10-26T18:06:18.543232Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:18.543342Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:18.542309Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:18.542382Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:18.542632Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-26T18:06:18.577017Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 5 to 5
2025-10-26T18:06:18.579192Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:18.579234Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 5
2025-10-26T18:06:18.579720Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:18.597609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 6 to 6
2025-10-26T18:06:18.623081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:18.623136Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 6
2025-10-26T18:06:18.617191Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:18.617244Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:18.618102Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 0 messageNo: 1 size: 511959
2025-10-26T18:06:18.618610Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 1 messageNo: 1 size: 511959
2025-10-26T18:06:18.619102Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 2 messageNo: 1 size: 511959
2025-10-26T18:06:18.619582Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 3 messageNo: 1 size: 511959
2025-10-26T18:06:18.620758Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 4 messageNo: 1 size: 205039
2025-10-26T18:06:18.620789Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 4 messageNo: 1 size 205039 offset: -1
2025-10-26T18:06:18.632667Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 0
2025-10-26T18:06:18.632729Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 1
2025-10-26T18:06:18.632743Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 2
2025-10-26T18:06:18.632763Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 3
2025-10-26T18:06:18.632781Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 4
2025-10-26T18:06:18.655043Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:18.700178Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:18.734966Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:18.738334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 7 to 7
2025-10-26T18:06:18.740524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:18.740564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 7
2025-10-26T18:06:18.741064Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:18.742283Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:18.742320Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:18.769027Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 1 partNo 4 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2253229 count 1 nextOffset 1 batches 5
2025-10-26T18:06:18.773380Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00004| size 2253219 WTime 1761501978773
2025-10-26T18:06:18.788594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 8 to 8
2025-10-26T18:06:18.794922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:18.794965Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 8
2025-10-26T18:06:18.798017Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:18.800027Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 2  requestId: <none> cookie: 2
2025-10-26T18:06:18.800552Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:18.820520Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:18.820563Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:18.821107Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 0 messageNo: 3 size: 511959
2025-10-26T18:06:18.821638Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 1 messageNo: 3 size: 511959
2025-10-26T18:06:18.829525Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 2 messageNo: 3 size: 511959
2025-10-26T18:06:18.830083Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 3 messageNo: 3 size: 511959
2025-10-26T18:06:18.830330Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 4 messageNo: 3 size: 205039
2025-10-26T18:06:18.830348Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 4 messageNo: 3 size 205039 offset: -1
2025-10-26T18:06:18.883527Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:18.879879Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:18.903508Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:18.905887Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:18.905926Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:18.906007Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 4  requestId: <none> cookie: 3
2025-10-26T18:06:18.907695Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:18.915730Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252910 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:18.915766Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:18.915826Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 0, Offset: 0 is stored on disk
2025-10-26T18:06:18.915836Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:18.915851Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 1, Offset: 0 is stored on disk
2025-10-26T18:06:18.915857Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:18.915872Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 2, Offset: 0 is stored on disk
2025-10-26T18:06:18.915879Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:18.915895Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 3, Offset: 0 is stored on disk
2025-10-26T18:06:18.915917Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:18.915931Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 4, Offset: 0 is stored on disk
2025-10-26T18:06:18.931624Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-26T18:06:18.931662Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-26T18:06:18.931811Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 0
2025-10-26T18:06:18.931842Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 1
2025-10-26T18:06:18.931859Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 2
2025-10-26T18:06:18.931877Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 3
2025-10-26T18:06:18.931892Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 4
2025-10-26T18:06:18.943187Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 9 to 9
2025-10-26T18:06:18.945250Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:18.945284Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 9
2025-10-26T18:06:18.946167Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:18.951071Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 0 EndOffset 0 newEndOffset 1 commitOffset 0 clientCommitOffset 0 cookie 1 readingFinished 0 firstRead 1
2025-10-26T18:06:18.951106Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 0 endOffset 1
2025-10-26T18:06:18.951416Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 0, endOffset# 1, WTime# 1761501976758, sizeLag# 2253219
2025-10-26T18:06:18.951433Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1TEvPartitionReady. Aval parts: 1
2025-10-26T18:06:18.951480Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 performing read request: guid# 7cae4254-498ef53e-78eb9bf-15b34856, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 1, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-26T18:06:18.951542Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 1 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 0 EndOffset 1 ClientCommitOffset 0 committedOffset 0 Guid 7cae4254-498ef53e-78eb9bf-15b34856
2025-10-26T18:06:18.958648Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: balancer: [72075186224037893] topic rt3.dc1--test-topic consumer user Reading of the partition 0 was started by user.
2025-10-26T18:06:18.958751Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:18.967793Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:18.984735Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 10 to 10
2025-10-26T18:06:18.987505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:18.987550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 10
2025-10-26T18:06:18.987582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528010[B] reached maximum (20971520[B])
2025-10-26T18:06:18.988036Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:19.031564Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 2 partNo 4 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 2253229 count 1 nextOffset 2 batches 5
2025-10-26T18:06:19.032735Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 1,1 HeadOffset 0 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00004| size 2253219 WTime 1761501979032
2025-10-26T18:06:19.038170Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1  requestId: <none> cookie: 1
2025-10-26T18:06:19.041070Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.041103Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.041661Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 0 messageNo: 5 size: 511959
2025-10-26T18:06:19.042197Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 1 messageNo: 5 size: 511959
2025-10-26T18:06:19.042213Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.042722Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 2 messageNo: 5 size: 511959
2025-10-26T18:06:19.043501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 1 offsets: 0 already_written: false write_statistics { persist_duration_ms: 282 queued_in_partition_duration_ms: 11 }
2025-10-26T18:06:19.043553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 1
2025-10-26T18:06:19.043584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275209[B]
2025-10-26T18:06:19.043232Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 3 messageNo: 5 size: 511959
2025-10-26T18:06:19.043475Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 4 messageNo: 5 size: 205039
2025-10-26T18:06:19.043495Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 4 messageNo: 5 size 205039 offset: -1
2025-10-26T18:06:19.045726Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.045773Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.045872Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.045881Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.045924Z node 2 :PERSQUEUE DEBUG: Read proxy: bootstrap for direct read id: 0
2025-10-26T18:06:19.046028Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:19.054087Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 11 to 11
2025-10-26T18:06:19.060383Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:19.060427Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 11
2025-10-26T18:06:19.060455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528011[B] reached maximum (20971520[B])
2025-10-26T18:06:19.061604Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:19.085660Z node 2 :PERSQUEUE DEBUG: read cookie 0 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-10-26T18:06:19.085865Z node 2 :PERSQUEUE DEBUG: read cookie 0 added 0 blobs, size 0 count 0 last offset 0
2025-10-26T18:06:19.088079Z node 2 :PERSQUEUE DEBUG: Reading cookie 0. All data is from uncompacted head.
2025-10-26T18:06:19.088130Z node 2 :PERSQUEUE DEBUG: FormAnswer 0
2025-10-26T18:06:19.090732Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1761501978632 queuesize 0 startOffset 0
2025-10-26T18:06:19.090912Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 6  requestId: <none> cookie: 4
2025-10-26T18:06:19.091046Z node 2 :PERSQUEUE DEBUG: read cookie 1 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 524288 endOffset 1 max time lag 0ms effective offset 0
2025-10-26T18:06:19.091056Z node 2 :PERSQUEUE DEBUG: read cookie 1 added 0 blobs, size 0 count 0 last offset 0
2025-10-26T18:06:19.093768Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.094518Z node 2 :PERSQUEUE DEBUG: Reading cookie 1. All data is from uncompacted head.
2025-10-26T18:06:19.094556Z node 2 :PERSQUEUE DEBUG: FormAnswer 0
2025-10-26T18:06:19.106956Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-26T18:06:19.110055Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:19.111299Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.111337Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.112111Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 0 messageNo: 7 size: 511959
2025-10-26T18:06:19.112575Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 1 messageNo: 7 size: 511959
2025-10-26T18:06:19.113317Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 2 messageNo: 7 size: 511959
2025-10-26T18:06:19.114149Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 3 messageNo: 7 size: 511959
2025-10-26T18:06:19.114636Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 4 messageNo: 7 size: 205039
2025-10-26T18:06:19.114652Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 4 messageNo: 7 size 205039 offset: -1
2025-10-26T18:06:19.126471Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:19.130463Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.130502Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.130542Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: __TMP__REQUEST__MARKER__
2025-10-26T18:06:19.130552Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.130742Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252910 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:19.130763Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.130793Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 0, Offset: 1 is stored on disk
2025-10-26T18:06:19.130803Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.130819Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 1, Offset: 1 is stored on disk
2025-10-26T18:06:19.130825Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.130843Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 2, Offset: 1 is stored on disk
2025-10-26T18:06:19.130851Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.130866Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 3, Offset: 1 is stored on disk
2025-10-26T18:06:19.130885Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.130899Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 4, Offset: 1 is stored on disk
2025-10-26T18:06:19.134487Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 0
2025-10-26T18:06:19.134530Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 1
2025-10-26T18:06:19.134543Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 2
2025-10-26T18:06:19.134558Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 3
2025-10-26T18:06:19.134572Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 4
2025-10-26T18:06:19.161164Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 3 partNo 4 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 2253229 count 1 nextOffset 3 batches 5
2025-10-26T18:06:19.161299Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 0
2025-10-26T18:06:19.161330Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 1
2025-10-26T18:06:19.161344Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 2
2025-10-26T18:06:19.161364Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 3
2025-10-26T18:06:19.197315Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 8  requestId: <none> cookie: 5
2025-10-26T18:06:19.197421Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 3  requestId: <none> cookie: 2
2025-10-26T18:06:19.197710Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.197794Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.198714Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 4 partNo 3 result is x0000000000_00000000000000000000_00000_0000000003_00015 size 8295679
2025-10-26T18:06:19.202593Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 4
2025-10-26T18:06:19.202722Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000000_00000_0000000003_00015 new key d0000000000_00000000000000000000_00000_0000000003_00015 size 8295679 WTime 1761501979202
2025-10-26T18:06:19.208540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 2 offsets: 1 already_written: false write_statistics { persist_duration_ms: 198 queued_in_partition_duration_ms: 101 }
2025-10-26T18:06:19.208646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 2
2025-10-26T18:06:19.208778Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275210[B]
2025-10-26T18:06:19.226644Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.218174Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:19.226685Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.230617Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 0 messageNo: 9 size: 511959
2025-10-26T18:06:19.233049Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 1 messageNo: 9 size: 511959
2025-10-26T18:06:19.234009Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 2 messageNo: 9 size: 511959
2025-10-26T18:06:19.235416Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:19.235957Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 3 messageNo: 9 size: 511959
2025-10-26T18:06:19.236209Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 4 messageNo: 9 size: 205039
2025-10-26T18:06:19.236229Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 4 messageNo: 9 size 205039 offset: -1
2025-10-26T18:06:19.239833Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.239859Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.240551Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 4 partNo 4 FormedBlobsCount 1 NewHead: Offset 3 PartNo 3 PackedSize 717144 count 1 nextOffset 4 batches 2
2025-10-26T18:06:19.241576Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 3,1 HeadOffset 0 endOffset 2 curOffset 4 d0000000000_00000000000000000003_00003_0000000001_00001| size 717134 WTime 1761501979239
2025-10-26T18:06:19.243609Z node 2 :PERSQUEUE DEBUG: read cookie 2 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 26214400 endOffset 2 max time lag 0ms effective offset 0
2025-10-26T18:06:19.243628Z node 2 :PERSQUEUE DEBUG: read cookie 2 added 0 blobs, size 0 count 0 last offset 0
2025-10-26T18:06:19.247820Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:19.248278Z node 2 :PERSQUEUE DEBUG: Reading cookie 2. All data is from uncompacted head.
2025-10-26T18:06:19.248312Z node 2 :PERSQUEUE DEBUG: FormAnswer 0
2025-10-26T18:06:19.249932Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: __TMP__REQUEST__MARKER__ cookie: 0
2025-10-26T18:06:19.249996Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 10  requestId: <none> cookie: 6
2025-10-26T18:06:19.251233Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.279158Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 12 to 12
2025-10-26T18:06:19.285581Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:19.286846Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:19.286888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 12
2025-10-26T18:06:19.286914Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528012[B] reached maximum (20971520[B])
2025-10-26T18:06:19.287430Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:19.316412Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.316498Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.318961Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 0 messageNo: 11 size: 511959
2025-10-26T18:06:19.321366Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 1 messageNo: 11 size: 511959
2025-10-26T18:06:19.321312Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:19.324409Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 2 Result { Offset: 0 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 1 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1761501978632 CreateTimestampMS: 1761501978419 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 0 SizeLag: 4301353 RealReadOffset: 0 WaitQuotaTimeMs: 0 } Cookie: 0 }
2025-10-26T18:06:19.333842Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:0:0
2025-10-26T18:06:19.325176Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 2 messageNo: 11 size: 511959
2025-10-26T18:06:19.325712Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 3 messageNo: 11 size: 511959
2025-10-26T18:06:19.337775Z :INFO: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1006 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-26T18:06:19.325945Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 4 messageNo: 11 size: 205039
2025-10-26T18:06:19.325966Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 4 messageNo: 11 size 205039 offset: -1
2025-10-26T18:06:19.330762Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 3 size 8295679
2025-10-26T18:06:19.343068Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 1 endOffset 2
2025-10-26T18:06:19.346019Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.346060Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.346377Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 12  requestId: <none> cookie: 7
2025-10-26T18:06:19.346984Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 2 ReadOffset 1 ReadGuid 7cae4254-498ef53e-78eb9bf-15b34856 has messages 1
2025-10-26T18:06:19.347487Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 1, endOffset# 2, WTime# 1761501978632, sizeLag# 4301353
2025-10-26T18:06:19.347506Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1TEvPartitionReady. Aval parts: 0
2025-10-26T18:06:19.347566Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 read done: guid# 7cae4254-498ef53e-78eb9bf-15b34856, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 2252976
2025-10-26T18:06:19.347601Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 response to read: guid# 7cae4254-498ef53e-78eb9bf-15b34856
2025-10-26T18:06:19.349342Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.352124Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Process answer. Aval parts: 1
2025-10-26T18:06:19.364996Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Got ReadResponse, serverBytesSize = 2252976, now ReadSizeBudget = 0, ReadSizeServerDelta = -1728688
2025-10-26T18:06:19.373905Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (0-0)
2025-10-26T18:06:19.373987Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 2252976 to budget
2025-10-26T18:06:19.404307Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.404356Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.404899Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 0 messageNo: 13 size: 511959
2025-10-26T18:06:19.405397Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 1 messageNo: 13 size: 511959
2025-10-26T18:06:19.411288Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 2 messageNo: 13 size: 511959
2025-10-26T18:06:19.411827Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 3 messageNo: 13 size: 511959
2025-10-26T18:06:19.412088Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 4 messageNo: 13 size: 205039
2025-10-26T18:06:19.412107Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 4 messageNo: 13 size 205039 offset: -1
2025-10-26T18:06:19.419615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593391036007794:8439] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-26T18:06:19.423773Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:19.428848Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:19.430141Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.430181Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.430778Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 14  requestId: <none> cookie: 8
2025-10-26T18:06:19.437796Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.442973Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 0 count 3 size 8295679 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:19.443122Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505820 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:19.445507Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.445556Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 0, Offset: 2 is stored on disk
2025-10-26T18:06:19.445569Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.445593Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 1, Offset: 2 is stored on disk
2025-10-26T18:06:19.445603Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.445622Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 2, Offset: 2 is stored on disk
2025-10-26T18:06:19.469185Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:19.445631Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.445650Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 3, Offset: 2 is stored on disk
2025-10-26T18:06:19.446826Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 size 8295679
2025-10-26T18:06:19.448121Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.448166Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 4, Offset: 2 is stored on disk
2025-10-26T18:06:19.448184Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.448204Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 0, Offset: 3 is stored on disk
2025-10-26T18:06:19.448213Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.448230Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 1, Offset: 3 is stored on disk
2025-10-26T18:06:19.448237Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.448257Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 2, Offset: 3 is stored on disk
2025-10-26T18:06:19.448265Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.448281Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 3, Offset: 3 is stored on disk
2025-10-26T18:06:19.448294Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.448311Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 4, Offset: 3 is stored on disk
2025-10-26T18:06:19.501364Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:19.504377Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.504422Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.504962Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 0 messageNo: 15 size: 511959
2025-10-26T18:06:19.505499Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 1 messageNo: 15 size: 511959
2025-10-26T18:06:19.510079Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 0
2025-10-26T18:06:19.510130Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 1
2025-10-26T18:06:19.510146Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 2
2025-10-26T18:06:19.510170Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 3
2025-10-26T18:06:19.510186Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 4
2025-10-26T18:06:19.512704Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 2 messageNo: 15 size: 511959
2025-10-26T18:06:19.513212Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 3 messageNo: 15 size: 511959
2025-10-26T18:06:19.513472Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 4 messageNo: 15 size: 205039
2025-10-26T18:06:19.513494Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 4 messageNo: 15 size 205039 offset: -1
2025-10-26T18:06:19.515130Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 5  requestId: <none> cookie: 3
2025-10-26T18:06:19.515224Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 7  requestId: <none> cookie: 4
2025-10-26T18:06:19.515280Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.515296Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.515989Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.516077Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.519460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 3 offsets: 2 already_written: false write_statistics { persist_duration_ms: 311 queued_in_partition_duration_ms: 49 }
2025-10-26T18:06:19.519511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 3
2025-10-26T18:06:19.519565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275211[B]
2025-10-26T18:06:19.523376Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 4 offsets: 3 already_written: false write_statistics { persist_duration_ms: 287 queued_in_partition_duration_ms: 28 }
2025-10-26T18:06:19.523431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 4
2025-10-26T18:06:19.531432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 13 to 13
2025-10-26T18:06:19.533546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:19.533589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 13
2025-10-26T18:06:19.550505Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 5 partNo 4 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 2253229 count 1 nextOffset 5 batches 5
2025-10-26T18:06:19.550600Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 0
2025-10-26T18:06:19.550623Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 1
2025-10-26T18:06:19.550633Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 2
2025-10-26T18:06:19.550645Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 3
2025-10-26T18:06:19.550656Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 4
2025-10-26T18:06:19.572367Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 6 partNo 4 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 4506426 count 2 nextOffset 6 batches 9
2025-10-26T18:06:19.572496Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 0
2025-10-26T18:06:19.572524Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 1
2025-10-26T18:06:19.572540Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 2
2025-10-26T18:06:19.572554Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 3
2025-10-26T18:06:19.572570Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 4
2025-10-26T18:06:19.580047Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (0-0)
2025-10-26T18:06:19.580119Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-26T18:06:19.580184Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] In ContinueReadingDataImpl, ReadSizeBudget = 2252976, ReadSizeServerDelta = -1728688
2025-10-26T18:06:19.580489Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-26T18:06:19.580534Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 0 ProducerId: "src_id" SeqNo: 1 CreateTime: 2025-10-26T18:06:18.419000Z WriteTime: 2025-10-26T18:06:18.632000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:50340", "_ip": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:19.592601Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 1, data# { read_request { bytes_size: 2252976 } }
2025-10-26T18:06:19.592762Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 got read request: guid# 11e9bb8b-d0d4bfd6-a4984bdb-79d221f7
2025-10-26T18:06:19.592825Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 performing read request: guid# b7df498e-39844b5d-507ecb3d-8fcd6d45, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 1, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-26T18:06:19.592893Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 1 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 1 EndOffset 2 ClientCommitOffset 0 committedOffset 0 Guid b7df498e-39844b5d-507ecb3d-8fcd6d45
2025-10-26T18:06:19.593321Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.593357Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.593423Z node 2 :PERSQUEUE DEBUG: Read proxy: bootstrap for direct read id: 0
2025-10-26T18:06:19.606449Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 14 to 14
2025-10-26T18:06:19.608639Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:19.608683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 14
2025-10-26T18:06:19.608709Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528014[B] reached maximum (20971520[B])
2025-10-26T18:06:19.609616Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-26T18:06:19.605473Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 7 partNo 4 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 6759623 count 3 nextOffset 7 batches 13
2025-10-26T18:06:19.606453Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 4,3 HeadOffset 3 endOffset 4 curOffset 7 d0000000000_00000000000000000004_00000_0000000003_00012| size 6759613 WTime 1761501979606
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:19.624345Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:19.626984Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 16  requestId: <none> cookie: 9
2025-10-26T18:06:19.627102Z node 2 :PERSQUEUE DEBUG: read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 1 count 1 size 524288 endOffset 4 max time lag 0ms effective offset 1
2025-10-26T18:06:19.627282Z node 2 :PERSQUEUE DEBUG: read cookie 3 added 1 blobs, size 0 count 2 last offset 2
2025-10-26T18:06:19.629959Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.650439Z node 2 :PERSQUEUE DEBUG: Reading cookie 3. Send blob request.
2025-10-26T18:06:19.653436Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.653473Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.654279Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 0 messageNo: 17 size: 511959
2025-10-26T18:06:19.655374Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 1 messageNo: 17 size: 511959
2025-10-26T18:06:19.655872Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 2 messageNo: 17 size: 511959
2025-10-26T18:06:19.664486Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:19.660665Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 3 messageNo: 17 size: 511959
2025-10-26T18:06:19.670133Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 4 messageNo: 17 size: 205039
2025-10-26T18:06:19.670175Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 4 messageNo: 17 size 205039 offset: -1
2025-10-26T18:06:19.674731Z node 2 :PERSQUEUE DEBUG: Got data from cache. Partition 0 offset 0 count 3 source 1 size 8295679 accessed 0 times before, last time 2025-10-26T18:06:19.000000Z
2025-10-26T18:06:19.674793Z node 2 :PERSQUEUE DEBUG: Reading cookie 3. All 1 blobs are from cache.
2025-10-26T18:06:19.674867Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0
2025-10-26T18:06:19.675108Z node 2 :PERSQUEUE DEBUG: FormAnswer 1
2025-10-26T18:06:19.679412Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:19.680081Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.680116Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.713667Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 1 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-26T18:06:19.719501Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 1 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:19.721300Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 1 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:19.731260Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 1 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:19.732290Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 1 totakecount 3 count 1 size 205085 from pos 0 cbcount 1
2025-10-26T18:06:19.735247Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-26T18:06:19.735397Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 18  requestId: <none> cookie: 10
2025-10-26T18:06:19.737816Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.758339Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:19.773639Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.773676Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.774330Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 0 messageNo: 19 size: 511959
2025-10-26T18:06:19.774869Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 1 messageNo: 19 size: 511959
2025-10-26T18:06:19.775427Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 2 messageNo: 19 size: 511959
2025-10-26T18:06:19.776437Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 3 messageNo: 19 size: 511959
2025-10-26T18:06:19.776702Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 4 messageNo: 19 size: 205039
2025-10-26T18:06:19.776720Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 4 messageNo: 19 size 205039 offset: -1
2025-10-26T18:06:19.780848Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:19.806832Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 4 Result { Offset: 1 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 2 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1761501978931 CreateTimestampMS: 1761501978451 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 6759704 RealReadOffset: 1 WaitQuotaTimeMs: 0 } Cookie: 1 }
2025-10-26T18:06:19.816637Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 2 endOffset 4
2025-10-26T18:06:19.816694Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 4 ReadOffset 2 ReadGuid b7df498e-39844b5d-507ecb3d-8fcd6d45 has messages 1
2025-10-26T18:06:19.820715Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 2, endOffset# 4, WTime# 1761501978931, sizeLag# 6759704
2025-10-26T18:06:19.820753Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1TEvPartitionReady. Aval parts: 0
2025-10-26T18:06:19.820797Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 read done: guid# b7df498e-39844b5d-507ecb3d-8fcd6d45, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 2252978
2025-10-26T18:06:19.820825Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 response to read: guid# b7df498e-39844b5d-507ecb3d-8fcd6d45
2025-10-26T18:06:19.822425Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 6758730 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:19.822458Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822494Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 0, Offset: 4 is stored on disk
2025-10-26T18:06:19.822504Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822533Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 1, Offset: 4 is stored on disk
2025-10-26T18:06:19.822542Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822562Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 2, Offset: 4 is stored on disk
2025-10-26T18:06:19.822569Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822584Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 3, Offset: 4 is stored on disk
2025-10-26T18:06:19.822604Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822622Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 4, Offset: 4 is stored on disk
2025-10-26T18:06:19.822630Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822645Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 0, Offset: 5 is stored on disk
2025-10-26T18:06:19.822653Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822669Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 1, Offset: 5 is stored on disk
2025-10-26T18:06:19.822676Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822692Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 2, Offset: 5 is stored on disk
2025-10-26T18:06:19.822698Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822714Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 3, Offset: 5 is stored on disk
2025-10-26T18:06:19.822723Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822739Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 4, Offset: 5 is stored on disk
2025-10-26T18:06:19.822746Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822762Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 0, Offset: 6 is stored on disk
2025-10-26T18:06:19.822769Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822784Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 1, Offset: 6 is stored on disk
2025-10-26T18:06:19.822791Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822808Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 2, Offset: 6 is stored on disk
2025-10-26T18:06:19.822815Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822831Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 3, Offset: 6 is stored on disk
2025-10-26T18:06:19.822840Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:19.822858Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 4, Offset: 6 is stored on disk
2025-10-26T18:06:19.834306Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.834353Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.834445Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 9  requestId: <none> cookie: 5
2025-10-26T18:06:19.834523Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 11  requestId: <none> cookie: 6
2025-10-26T18:06:19.834587Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 13  requestId: <none> cookie: 7
2025-10-26T18:06:19.850055Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 5 offsets: 4 already_written: false write_statistics { persist_duration_ms: 312 queued_in_partition_duration_ms: 261 }
2025-10-26T18:06:19.850119Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 5
2025-10-26T18:06:19.850171Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275213[B]
2025-10-26T18:06:19.839208Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Process answer. Aval parts: 1
2025-10-26T18:06:19.840261Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.840338Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.840371Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.851984Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 6 offsets: 5 already_written: false write_statistics { persist_duration_ms: 272 queued_in_partition_duration_ms: 217 }
2025-10-26T18:06:19.852053Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 6
2025-10-26T18:06:19.852487Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Got ReadResponse, serverBytesSize = 2252978, now ReadSizeBudget = 0, ReadSizeServerDelta = -1728690
2025-10-26T18:06:19.853852Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (1-1)
2025-10-26T18:06:19.853885Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 2252978 to budget
2025-10-26T18:06:19.856613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 15 to 15
2025-10-26T18:06:19.858814Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:19.858851Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 15
2025-10-26T18:06:19.860798Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 7 offsets: 6 already_written: false write_statistics { persist_duration_ms: 250 queued_in_partition_duration_ms: 163 }
2025-10-26T18:06:19.860845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 7
2025-10-26T18:06:19.866060Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 0
2025-10-26T18:06:19.866117Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 1
2025-10-26T18:06:19.888639Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (1-1)
2025-10-26T18:06:19.888697Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-26T18:06:19.888723Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] In ContinueReadingDataImpl, ReadSizeBudget = 2252978, ReadSizeServerDelta = -1728690
2025-10-26T18:06:19.888968Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-26T18:06:19.889004Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 1 ProducerId: "src_id" SeqNo: 2 CreateTime: 2025-10-26T18:06:18.451000Z WriteTime: 2025-10-26T18:06:18.931000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:19.890202Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 1, data# { read_request { bytes_size: 2252978 } }
2025-10-26T18:06:19.890321Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 got read request: guid# d6769479-48b5d311-12ea33a-be7c1d77
2025-10-26T18:06:19.890361Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 performing read request: guid# 8636ee8e-a23ee30f-ce5a159e-14d239fe, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 2, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-26T18:06:19.890416Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 2 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 2 EndOffset 4 ClientCommitOffset 0 committedOffset 0 Guid 8636ee8e-a23ee30f-ce5a159e-14d239fe
2025-10-26T18:06:19.892840Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:19.892879Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:19.892958Z node 2 :PERSQUEUE DEBUG: Read proxy: bootstrap for direct read id: 0
2025-10-26T18:06:19.897639Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 8 partNo 1 result is x0000000000_00000000000000000003_00003_0000000004_00014 size 7988778
2025-10-26T18:06:19.900637Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 2
2025-10-26T18:06:19.900690Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 3
2025-10-26T18:06:19.900710Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 4
2025-10-26T18:06:19.900769Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000003_00003_0000000004_00014 new key d0000000000_00000000000000000003_00003_0000000004_00014 size 7988778 WTime 1761501979900
2025-10-26T18:06:19.913045Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 16 to 16
2025-10-26T18:06:19.916023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:19.916066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 16
2025-10-26T18:06:19.916632Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:19.919896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593391036007794:8439] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=1 Status=SUCCESS
2025-10-26T18:06:19.919922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: TPartitionChooser [1:7565593391036007794:8439] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-26T18:06:19.926923Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 8 partNo 4 FormedBlobsCount 1 NewHead: Offset 7 PartNo 1 PackedSize 1741198 count 1 nextOffset 8 batches 4
2025-10-26T18:06:19.927043Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 0
2025-10-26T18:06:19.927069Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 1
2025-10-26T18:06:19.927085Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 2
2025-10-26T18:06:19.927100Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 3
2025-10-26T18:06:19.927119Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 4
2025-10-26T18:06:19.930505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 17 to 17
2025-10-26T18:06:19.933023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:19.933066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 17
2025-10-26T18:06:19.933094Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528017[B] reached maximum (20971520[B])
2025-10-26T18:06:19.934082Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:19.940909Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 9 partNo 4 FormedBlobsCount 0 NewHead: Offset 7 PartNo 1 PackedSize 3994395 count 2 nextOffset 9 batches 8
2025-10-26T18:06:19.941027Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 0
2025-10-26T18:06:19.941057Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 1
2025-10-26T18:06:19.941071Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 2
2025-10-26T18:06:19.941086Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 3
2025-10-26T18:06:19.941100Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 4
2025-10-26T18:06:19.960383Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 10 partNo 4 FormedBlobsCount 0 NewHead: Offset 7 PartNo 1 PackedSize 6247592 count 3 nextOffset 10 batches 12
2025-10-26T18:06:19.961312Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 7,3 HeadOffset 3 endOffset 7 curOffset 10 d0000000000_00000000000000000007_00001_0000000003_00011| size 6247582 WTime 1761501979960
2025-10-26T18:06:19.974530Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:19.977777Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 20  requestId: <none> cookie: 11
2025-10-26T18:06:19.977920Z node 2 :PERSQUEUE DEBUG: read cookie 4 Topic 'rt3.dc1--test-topic' partition 0 user user offset 2 count 2 size 524288 endOffset 7 max time lag 0ms effective offset 2
2025-10-26T18:06:19.978040Z node 2 :PERSQUEUE DEBUG: read cookie 4 added 1 blobs, size 0 count 1 last offset 3
2025-10-26T18:06:19.978233Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:19.981828Z node 2 :PERSQUEUE DEBUG: Reading cookie 4. Send blob request.
2025-10-26T18:06:20.007638Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:20.015586Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 3 count 4 size 7988778
2025-10-26T18:06:20.025202Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.025257Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.031449Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 0 messageNo: 21 size: 511959
2025-10-26T18:06:20.037069Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:20.037257Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 1 messageNo: 21 size: 511959
2025-10-26T18:06:20.038255Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 2 messageNo: 21 size: 511959
2025-10-26T18:06:20.039461Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 3 messageNo: 21 size: 511959
2025-10-26T18:06:20.039892Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 4 messageNo: 21 size: 205040
2025-10-26T18:06:20.039930Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 4 messageNo: 21 size 205040 offset: -1
2025-10-26T18:06:20.041908Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.041935Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.042043Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 22  requestId: <none> cookie: 12
2025-10-26T18:06:20.043458Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.060863Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:20.065844Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.065893Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.067169Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 0 messageNo: 23 size: 511959
2025-10-26T18:06:20.070017Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 1 messageNo: 23 size: 511959
2025-10-26T18:06:20.070153Z node 2 :PERSQUEUE DEBUG: Got data from cache. Partition 0 offset 0 count 3 source 1 size 8295679 accessed 1 times before, last time 2025-10-26T18:06:19.000000Z
2025-10-26T18:06:20.070208Z node 2 :PERSQUEUE DEBUG: Reading cookie 4. All 1 blobs are from cache.
2025-10-26T18:06:20.070351Z node 2 :PERSQUEUE DEBUG: FormAnswer 1
2025-10-26T18:06:20.074211Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0
2025-10-26T18:06:20.083510Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:20.084140Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 2 messageNo: 23 size: 511959
2025-10-26T18:06:20.090285Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 3 messageNo: 23 size: 511959
2025-10-26T18:06:20.090970Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 4 messageNo: 23 size: 205040
2025-10-26T18:06:20.091000Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 4 messageNo: 23 size 205040 offset: -1
2025-10-26T18:06:20.136131Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 2 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-26T18:06:20.142408Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.142451Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.149916Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 2 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.152474Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 2 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.154290Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 2 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.155382Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 2 totakecount 3 count 1 size 205085 from pos 0 cbcount 1
2025-10-26T18:06:20.166292Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 3 totakecount 3 count 0 size 1536000 from pos 0 cbcount 3
2025-10-26T18:06:20.168367Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 3 count 4 size 7988778 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:20.168495Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 6758730 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:20.168512Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168544Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 0, Offset: 7 is stored on disk
2025-10-26T18:06:20.168555Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168572Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 1, Offset: 7 is stored on disk
2025-10-26T18:06:20.168580Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168598Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 2, Offset: 7 is stored on disk
2025-10-26T18:06:20.168606Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168621Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 3, Offset: 7 is stored on disk
2025-10-26T18:06:20.168641Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168657Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 4, Offset: 7 is stored on disk
2025-10-26T18:06:20.168665Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168680Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 0, Offset: 8 is stored on disk
2025-10-26T18:06:20.168686Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168703Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 1, Offset: 8 is stored on disk
2025-10-26T18:06:20.168711Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168727Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 2, Offset: 8 is stored on disk
2025-10-26T18:06:20.168733Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168749Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 3, Offset: 8 is stored on disk
2025-10-26T18:06:20.168758Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168775Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 4, Offset: 8 is stored on disk
2025-10-26T18:06:20.168782Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168798Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 0, Offset: 9 is stored on disk
2025-10-26T18:06:20.168806Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168821Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 1, Offset: 9 is stored on disk
2025-10-26T18:06:20.168827Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168844Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 2, Offset: 9 is stored on disk
2025-10-26T18:06:20.168851Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168867Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 3, Offset: 9 is stored on disk
2025-10-26T18:06:20.168876Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.168894Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 4, Offset: 9 is stored on disk
2025-10-26T18:06:20.169775Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 size 7988778
2025-10-26T18:06:20.185466Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 2
2025-10-26T18:06:20.193371Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 0
2025-10-26T18:06:20.193444Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 1
2025-10-26T18:06:20.193465Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 2
2025-10-26T18:06:20.193508Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 3
2025-10-26T18:06:20.193546Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 4
2025-10-26T18:06:20.200959Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 24  requestId: <none> cookie: 13
2025-10-26T18:06:20.201066Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 15  requestId: <none> cookie: 8
2025-10-26T18:06:20.201123Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 17  requestId: <none> cookie: 9
2025-10-26T18:06:20.201177Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 19  requestId: <none> cookie: 10
2025-10-26T18:06:20.205783Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.205868Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.205911Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.205943Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.216371Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 8 offsets: 7 already_written: false write_statistics { persist_duration_ms: 299 queued_in_partition_duration_ms: 239 }
2025-10-26T18:06:20.216419Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 8
2025-10-26T18:06:20.216453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275216[B]
2025-10-26T18:06:20.217475Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:20.221817Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.227992Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:20.221866Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.222395Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 0 messageNo: 25 size: 511959
2025-10-26T18:06:20.222894Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 1 messageNo: 25 size: 511959
2025-10-26T18:06:20.228643Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 2 messageNo: 25 size: 511959
2025-10-26T18:06:20.229096Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 3 messageNo: 25 size: 511959
2025-10-26T18:06:20.229328Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 4 messageNo: 25 size: 205040
2025-10-26T18:06:20.229349Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 4 messageNo: 25 size 205040 offset: -1
2025-10-26T18:06:20.233050Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 7 Result { Offset: 2 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 3 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1761501979134 CreateTimestampMS: 1761501978494 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 11983298 RealReadOffset: 2 WaitQuotaTimeMs: 0 } Cookie: 2 }
2025-10-26T18:06:20.234395Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.234428Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.239111Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 3 endOffset 7
2025-10-26T18:06:20.239163Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 7 ReadOffset 3 ReadGuid 8636ee8e-a23ee30f-ce5a159e-14d239fe has messages 1
2025-10-26T18:06:20.241361Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 3, endOffset# 7, WTime# 1761501979134, sizeLag# 11983298
2025-10-26T18:06:20.244391Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1TEvPartitionReady. Aval parts: 0
2025-10-26T18:06:20.248172Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 9 offsets: 8 already_written: false write_statistics { persist_duration_ms: 239 queued_in_partition_duration_ms: 255 }
2025-10-26T18:06:20.248207Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 9
2025-10-26T18:06:20.248766Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 10 offsets: 9 already_written: false write_statistics { persist_duration_ms: 225 queued_in_partition_duration_ms: 161 }
2025-10-26T18:06:20.248784Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 10
2025-10-26T18:06:20.251899Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 18 to 18
2025-10-26T18:06:20.258367Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:20.258413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 18
2025-10-26T18:06:20.259294Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:20.244476Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 read done: guid# 8636ee8e-a23ee30f-ce5a159e-14d239fe, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 2252977
2025-10-26T18:06:20.263621Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Got ReadResponse, serverBytesSize = 2252977, now ReadSizeBudget = 0, ReadSizeServerDelta = -1728689
2025-10-26T18:06:20.263922Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (2-2)
2025-10-26T18:06:20.263965Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 2252977 to budget
2025-10-26T18:06:20.244509Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 response to read: guid# 8636ee8e-a23ee30f-ce5a159e-14d239fe
2025-10-26T18:06:20.252929Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Process answer. Aval parts: 1
2025-10-26T18:06:20.283058Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 11 partNo 4 result is x0000000000_00000000000000000007_00001_0000000003_00015 size 8295579
2025-10-26T18:06:20.283138Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000007_00001_0000000003_00015 new key d0000000000_00000000000000000007_00001_0000000003_00015 size 8295579 WTime 1761501980281
2025-10-26T18:06:20.293026Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 19 to 19
2025-10-26T18:06:20.295776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:20.295819Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 19
2025-10-26T18:06:20.296556Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (2-2)
2025-10-26T18:06:20.296603Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-26T18:06:20.296632Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] In ContinueReadingDataImpl, ReadSizeBudget = 2252977, ReadSizeServerDelta = -1728689
2025-10-26T18:06:20.296769Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-26T18:06:20.296799Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 2 ProducerId: "src_id" SeqNo: 3 CreateTime: 2025-10-26T18:06:18.494000Z WriteTime: 2025-10-26T18:06:19.134000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:20.304805Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 11 partNo 4 FormedBlobsCount 1 NewHead: Offset 10 PartNo 4 PackedSize 205118 count 1 nextOffset 11 batches 1
2025-10-26T18:06:20.307608Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 0
2025-10-26T18:06:20.307750Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 1
2025-10-26T18:06:20.307771Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 2
2025-10-26T18:06:20.307788Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 3
2025-10-26T18:06:20.307803Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 4
2025-10-26T18:06:20.313512Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.313544Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.313602Z node 2 :PERSQUEUE DEBUG: Read proxy: bootstrap for direct read id: 0
2025-10-26T18:06:20.309826Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 1, data# { read_request { bytes_size: 2252977 } }
2025-10-26T18:06:20.310045Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 got read request: guid# 7ba6a489-b18bafa8-9b42011d-1127b359
2025-10-26T18:06:20.310096Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 performing read request: guid# 2ebdc82c-433da182-d719ce65-c75947b3, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 4, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-26T18:06:20.310157Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 4 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 3 EndOffset 7 ClientCommitOffset 0 committedOffset 0 Guid 2ebdc82c-433da182-d719ce65-c75947b3
2025-10-26T18:06:20.323266Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 20 to 20
2025-10-26T18:06:20.331556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:20.331600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 20
2025-10-26T18:06:20.331626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:20.332374Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:20.340722Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 12 partNo 4 FormedBlobsCount 0 NewHead: Offset 10 PartNo 4 PackedSize 2458316 count 2 nextOffset 12 batches 5
2025-10-26T18:06:20.341665Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 10,2 HeadOffset 7 endOffset 10 curOffset 12 d0000000000_00000000000000000010_00004_0000000002_00004| size 2458306 WTime 1761501980341
2025-10-26T18:06:20.352029Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:20.357533Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 26  requestId: <none> cookie: 14
2025-10-26T18:06:20.357700Z node 2 :PERSQUEUE DEBUG: read cookie 5 Topic 'rt3.dc1--test-topic' partition 0 user user offset 3 count 4 size 524288 endOffset 10 max time lag 0ms effective offset 3
2025-10-26T18:06:20.357939Z node 2 :PERSQUEUE DEBUG: read cookie 5 added 2 blobs, size 7988778 count 4 last offset 7
2025-10-26T18:06:20.360348Z node 2 :PERSQUEUE DEBUG: Reading cookie 5. Send blob request.
2025-10-26T18:06:20.361779Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.372688Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 1 count 3 size 8295579
2025-10-26T18:06:20.393518Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:20.396108Z node 2 :PERSQUEUE DEBUG: Got data from cache. Partition 0 offset 0 count 3 source 1 size 8295679 accessed 2 times before, last time 2025-10-26T18:06:20.000000Z
2025-10-26T18:06:20.396165Z node 2 :PERSQUEUE DEBUG: Got data from cache. Partition 0 offset 3 count 4 source 1 size 7988778 accessed 0 times before, last time 2025-10-26T18:06:20.000000Z
2025-10-26T18:06:20.396207Z node 2 :PERSQUEUE DEBUG: Reading cookie 5. All 2 blobs are from cache.
2025-10-26T18:06:20.396285Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0
2025-10-26T18:06:20.396362Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3
2025-10-26T18:06:20.399559Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:20.400169Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.400229Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.400814Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 0 messageNo: 27 size: 511959
2025-10-26T18:06:20.401333Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 1 messageNo: 27 size: 511959
2025-10-26T18:06:20.402043Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 2 messageNo: 27 size: 511959
2025-10-26T18:06:20.402541Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 3 messageNo: 27 size: 511959
2025-10-26T18:06:20.402797Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 4 messageNo: 27 size: 205040
2025-10-26T18:06:20.402817Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 4 messageNo: 27 size 205040 offset: -1
2025-10-26T18:06:20.404836Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.404858Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.405007Z node 2 :PERSQUEUE DEBUG: FormAnswer 2
2025-10-26T18:06:20.445228Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 3 totakecount 3 count 0 size 1536000 from pos 0 cbcount 3
2025-10-26T18:06:20.449246Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 3 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.450496Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 3 totakecount 4 count 1 size 205085 from pos 0 cbcount 1
2025-10-26T18:06:20.451884Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 4 totakecount 4 count 0 size 512009 from pos 0 cbcount 1
2025-10-26T18:06:20.453567Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 4 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.455244Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 4 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.456966Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 4 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.459212Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 4 totakecount 4 count 1 size 717094 from pos 0 cbcount 2
2025-10-26T18:06:20.461663Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 5 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.463518Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 5 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.465190Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 5 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.467441Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 5 totakecount 4 count 1 size 717094 from pos 0 cbcount 2
2025-10-26T18:06:20.469561Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 6 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.471045Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 6 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.472402Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 6 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:20.473268Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 6 totakecount 4 count 1 size 205085 from pos 0 cbcount 1
2025-10-26T18:06:20.481990Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 3
2025-10-26T18:06:20.539551Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 7 count 3 size 8295579 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:20.539817Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505822 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:20.539816Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 size 8295579
2025-10-26T18:06:20.539839Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.539873Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 0, Offset: 10 is stored on disk
2025-10-26T18:06:20.539885Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.539907Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 1, Offset: 10 is stored on disk
2025-10-26T18:06:20.539915Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.539934Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 2, Offset: 10 is stored on disk
2025-10-26T18:06:20.539941Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.539962Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 3, Offset: 10 is stored on disk
2025-10-26T18:06:20.539988Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.540009Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 4, Offset: 10 is stored on disk
2025-10-26T18:06:20.540017Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.540037Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 0, Offset: 11 is stored on disk
2025-10-26T18:06:20.540046Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.540068Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 1, Offset: 11 is stored on disk
2025-10-26T18:06:20.540077Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.540117Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 2, Offset: 11 is stored on disk
2025-10-26T18:06:20.540127Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.540146Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 3, Offset: 11 is stored on disk
2025-10-26T18:06:20.540157Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.540187Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 4, Offset: 11 is stored on disk
2025-10-26T18:06:20.549133Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 0
2025-10-26T18:06:20.549196Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 1
2025-10-26T18:06:20.549215Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 2
2025-10-26T18:06:20.549232Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 3
2025-10-26T18:06:20.549252Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 4
2025-10-26T18:06:20.584139Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 13 partNo 4 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 2253230 count 1 nextOffset 13 batches 5
2025-10-26T18:06:20.584283Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 0
2025-10-26T18:06:20.584320Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 1
2025-10-26T18:06:20.584338Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 2
2025-10-26T18:06:20.584357Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 3
2025-10-26T18:06:20.584375Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 4
2025-10-26T18:06:20.608159Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 10 Result { Offset: 3 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 4 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1761501979157 CreateTimestampMS: 1761501978533 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 4 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 5 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1761501979509 CreateTimestampMS: 1761501978568 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 5 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 6 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1761501979550 CreateTimestampMS: 1761501978595 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 6 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 7 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1761501979571 CreateTimestampMS: 1761501978696 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 2 SizeLag: 13826504 RealReadOffset: 6 WaitQuotaTimeMs: 0 } Cookie: 3 }
2025-10-26T18:06:20.612086Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 14 partNo 4 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 4506428 count 2 nextOffset 14 batches 9
2025-10-26T18:06:20.612809Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 12,2 HeadOffset 10 endOffset 12 curOffset 14 d0000000000_00000000000000000012_00000_0000000002_00008| size 4506418 WTime 1761501980610
2025-10-26T18:06:20.623147Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 28  requestId: <none> cookie: 15
2025-10-26T18:06:20.623269Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 21  requestId: <none> cookie: 11
2025-10-26T18:06:20.623331Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 23  requestId: <none> cookie: 12
2025-10-26T18:06:20.624801Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.624897Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.624941Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.638560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 11 offsets: 10 already_written: false write_statistics { persist_duration_ms: 346 queued_in_partition_duration_ms: 150 }
2025-10-26T18:06:20.638615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 11
2025-10-26T18:06:20.638647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:20.652176Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 21 to 21
2025-10-26T18:06:20.646449Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:20.662712Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.662758Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.663349Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 0 messageNo: 29 size: 511959
2025-10-26T18:06:20.663882Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 1 messageNo: 29 size: 511959
2025-10-26T18:06:20.664441Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 2 messageNo: 29 size: 511959
2025-10-26T18:06:20.664995Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 3 messageNo: 29 size: 511959
2025-10-26T18:06:20.665248Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 4 messageNo: 29 size: 205040
2025-10-26T18:06:20.665269Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 4 messageNo: 29 size 205040 offset: -1
2025-10-26T18:06:20.673199Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 7 endOffset 10
2025-10-26T18:06:20.673256Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 10 ReadOffset 7 ReadGuid 2ebdc82c-433da182-d719ce65-c75947b3 has messages 1
2025-10-26T18:06:20.715185Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 7, endOffset# 10, WTime# 1761501979571, sizeLag# 13826504
2025-10-26T18:06:20.715218Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1TEvPartitionReady. Aval parts: 0
2025-10-26T18:06:20.715286Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 read done: guid# 2ebdc82c-433da182-d719ce65-c75947b3, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 9011863
2025-10-26T18:06:20.715321Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 response to read: guid# 2ebdc82c-433da182-d719ce65-c75947b3
2025-10-26T18:06:20.723005Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:20.726440Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.726478Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.726517Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:20.727462Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 30  requestId: <none> cookie: 16
2025-10-26T18:06:20.730611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:20.730650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 21
2025-10-26T18:06:20.730677Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:20.731474Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:20.729978Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.760411Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 12 offsets: 11 already_written: false write_statistics { persist_duration_ms: 230 queued_in_partition_duration_ms: 141 }
2025-10-26T18:06:20.760466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 12
2025-10-26T18:06:20.760511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:20.775045Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.775105Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.781902Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:20.799024Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 0 messageNo: 31 size: 511959
2025-10-26T18:06:20.799629Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 1 messageNo: 31 size: 511959
2025-10-26T18:06:20.800546Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 2 messageNo: 31 size: 511959
2025-10-26T18:06:20.801131Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 3 messageNo: 31 size: 511959
2025-10-26T18:06:20.801371Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 4 messageNo: 31 size: 205040
2025-10-26T18:06:20.801390Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 4 messageNo: 31 size 205040 offset: -1
2025-10-26T18:06:20.792441Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:20.803012Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.803049Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.803278Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 32  requestId: <none> cookie: 17
2025-10-26T18:06:20.810079Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.821146Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.821209Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.829072Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Process answer. Aval parts: 1
2025-10-26T18:06:20.830181Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:20.850942Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 0 messageNo: 33 size: 511959
2025-10-26T18:06:20.851514Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 1 messageNo: 33 size: 511959
2025-10-26T18:06:20.852057Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 2 messageNo: 33 size: 511959
2025-10-26T18:06:20.852583Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 3 messageNo: 33 size: 511959
2025-10-26T18:06:20.853881Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 4 messageNo: 33 size: 205040
2025-10-26T18:06:20.853903Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 4 messageNo: 33 size 205040 offset: -1
2025-10-26T18:06:20.882127Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:20.889838Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:20.889902Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:20.890022Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 34  requestId: <none> cookie: 18
2025-10-26T18:06:20.897842Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.933913Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505822 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:20.933954Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.933992Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 0, Offset: 12 is stored on disk
2025-10-26T18:06:20.934005Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.934024Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 1, Offset: 12 is stored on disk
2025-10-26T18:06:20.934032Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.934052Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 2, Offset: 12 is stored on disk
2025-10-26T18:06:20.934062Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.934081Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 3, Offset: 12 is stored on disk
2025-10-26T18:06:20.934106Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.934124Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 4, Offset: 12 is stored on disk
2025-10-26T18:06:20.934130Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.934150Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 0, Offset: 13 is stored on disk
2025-10-26T18:06:20.934158Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.934179Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 1, Offset: 13 is stored on disk
2025-10-26T18:06:20.934185Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.934202Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 2, Offset: 13 is stored on disk
2025-10-26T18:06:20.934210Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.934231Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 3, Offset: 13 is stored on disk
2025-10-26T18:06:20.934240Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:20.934258Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 4, Offset: 13 is stored on disk
2025-10-26T18:06:20.964992Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 25  requestId: <none> cookie: 13
2025-10-26T18:06:20.965101Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 27  requestId: <none> cookie: 14
2025-10-26T18:06:20.967585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 22 to 22
2025-10-26T18:06:20.970270Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:20.970314Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 22
2025-10-26T18:06:20.970345Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:20.971156Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:20.974756Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:20.986395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 13 offsets: 12 already_written: false write_statistics { persist_duration_ms: 385 queued_in_partition_duration_ms: 197 }
2025-10-26T18:06:20.986459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 13
2025-10-26T18:06:20.986496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:20.974863Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.008086Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 0
2025-10-26T18:06:21.008153Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 1
2025-10-26T18:06:21.008173Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 2
2025-10-26T18:06:21.011105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 23 to 23
2025-10-26T18:06:21.011963Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:21.023888Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.023940Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.024450Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 0 messageNo: 35 size: 511959
2025-10-26T18:06:21.030470Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 1 messageNo: 35 size: 511959
2025-10-26T18:06:21.030957Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 2 messageNo: 35 size: 511959
2025-10-26T18:06:21.031453Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 3 messageNo: 35 size: 511959
2025-10-26T18:06:21.031716Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 4 messageNo: 35 size: 205040
2025-10-26T18:06:21.031741Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 4 messageNo: 35 size 205040 offset: -1
2025-10-26T18:06:21.034991Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:21.035039Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 23
2025-10-26T18:06:21.038936Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:21.041553Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.041606Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.050759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:21.052823Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:21.053194Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 14 offsets: 13 already_written: false write_statistics { persist_duration_ms: 351 queued_in_partition_duration_ms: 43 }
2025-10-26T18:06:21.053246Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 14
2025-10-26T18:06:21.053275Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:21.072892Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 24 to 24
2025-10-26T18:06:21.074984Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:21.075027Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 24
2025-10-26T18:06:21.075057Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:21.075626Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:21.079124Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Got ReadResponse, serverBytesSize = 9011863, now ReadSizeBudget = 0, ReadSizeServerDelta = -8487575
2025-10-26T18:06:21.079479Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (3-3)
2025-10-26T18:06:21.079549Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 9011863 to budget
2025-10-26T18:06:21.080876Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 15 partNo 2 result is x0000000000_00000000000000000010_00004_0000000004_00014 size 7988760
2025-10-26T18:06:21.089605Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 3
2025-10-26T18:06:21.089670Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 4
2025-10-26T18:06:21.089759Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000010_00004_0000000004_00014 new key d0000000000_00000000000000000010_00004_0000000004_00014 size 7988760 WTime 1761501981089
2025-10-26T18:06:21.095379Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 15 partNo 4 FormedBlobsCount 1 NewHead: Offset 14 PartNo 2 PackedSize 1229172 count 1 nextOffset 15 batches 3
2025-10-26T18:06:21.095507Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 0
2025-10-26T18:06:21.095546Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 1
2025-10-26T18:06:21.095570Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 2
2025-10-26T18:06:21.095588Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 3
2025-10-26T18:06:21.095604Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 4
2025-10-26T18:06:21.108652Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 16 partNo 4 FormedBlobsCount 0 NewHead: Offset 14 PartNo 2 PackedSize 3482370 count 2 nextOffset 16 batches 7
2025-10-26T18:06:21.108801Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 0
2025-10-26T18:06:21.108839Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 1
2025-10-26T18:06:21.108866Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 2
2025-10-26T18:06:21.108885Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 3
2025-10-26T18:06:21.108902Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 4
2025-10-26T18:06:21.131256Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 17 partNo 4 FormedBlobsCount 0 NewHead: Offset 14 PartNo 2 PackedSize 5735568 count 3 nextOffset 17 batches 11
2025-10-26T18:06:21.132180Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 14,3 HeadOffset 10 endOffset 14 curOffset 17 d0000000000_00000000000000000014_00002_0000000003_00010| size 5735558 WTime 1761501981132
2025-10-26T18:06:21.153105Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:21.153235Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 36  requestId: <none> cookie: 19
2025-10-26T18:06:21.157625Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.184756Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:21.190713Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.190769Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.191680Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 0 messageNo: 37 size: 511959
2025-10-26T18:06:21.192351Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 1 messageNo: 37 size: 511959
2025-10-26T18:06:21.193031Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 2 messageNo: 37 size: 511959
2025-10-26T18:06:21.198073Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 3 messageNo: 37 size: 511959
2025-10-26T18:06:21.198713Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 4 messageNo: 37 size: 205040
2025-10-26T18:06:21.198748Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 4 messageNo: 37 size 205040 offset: -1
2025-10-26T18:06:21.199556Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 4 count 4 size 7988760
2025-10-26T18:06:21.205318Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:21.206282Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.205863Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.205919Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.206021Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 38  requestId: <none> cookie: 20
2025-10-26T18:06:21.215305Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:21.217907Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.217962Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.218508Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 0 messageNo: 39 size: 511959
2025-10-26T18:06:21.222187Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:21.219440Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 1 messageNo: 39 size: 511959
2025-10-26T18:06:21.219986Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 2 messageNo: 39 size: 511959
2025-10-26T18:06:21.220528Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 3 messageNo: 39 size: 511959
2025-10-26T18:06:21.220772Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 4 messageNo: 39 size: 205040
2025-10-26T18:06:21.220795Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 4 messageNo: 39 size 205040 offset: -1
2025-10-26T18:06:21.223403Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.223439Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.223516Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 40  requestId: <none> cookie: 21
2025-10-26T18:06:21.223783Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.239663Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.239736Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.240423Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 0 messageNo: 41 size: 511959
2025-10-26T18:06:21.241021Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 1 messageNo: 41 size: 511959
2025-10-26T18:06:21.242682Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 2 messageNo: 41 size: 511959
2025-10-26T18:06:21.243202Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 3 messageNo: 41 size: 511959
2025-10-26T18:06:21.243577Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 4 messageNo: 41 size: 205040
2025-10-26T18:06:21.243623Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 4 messageNo: 41 size 205040 offset: -1
2025-10-26T18:06:21.239376Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:21.272835Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:21.323993Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.324055Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.324301Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 42  requestId: <none> cookie: 22
2025-10-26T18:06:21.324530Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.333415Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:21.342755Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:21.343334Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.343381Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.343963Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 0 messageNo: 43 size: 511959
2025-10-26T18:06:21.344508Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 1 messageNo: 43 size: 511959
2025-10-26T18:06:21.345182Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 2 messageNo: 43 size: 511959
2025-10-26T18:06:21.345713Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 3 messageNo: 43 size: 511959
2025-10-26T18:06:21.346173Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 4 messageNo: 43 size: 205040
2025-10-26T18:06:21.346204Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 4 messageNo: 43 size 205040 offset: -1
2025-10-26T18:06:21.359737Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.359778Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.359883Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 44  requestId: <none> cookie: 23
2025-10-26T18:06:21.361880Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.384576Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.384636Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.385151Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 0 messageNo: 45 size: 511959
2025-10-26T18:06:21.385746Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 1 messageNo: 45 size: 511959
2025-10-26T18:06:21.385806Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:21.388403Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 2 messageNo: 45 size: 511959
2025-10-26T18:06:21.389160Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 3 messageNo: 45 size: 511959
2025-10-26T18:06:21.389403Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 4 messageNo: 45 size: 205040
2025-10-26T18:06:21.389445Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 4 messageNo: 45 size 205040 offset: -1
2025-10-26T18:06:21.393159Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:21.394105Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.394156Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.394279Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 46  requestId: <none> cookie: 24
2025-10-26T18:06:21.396621Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.399777Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 10 count 4 size 7988760 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:21.399927Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 size 7988760
2025-10-26T18:06:21.400014Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 6758733 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:21.400075Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400146Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 0, Offset: 14 is stored on disk
2025-10-26T18:06:21.400196Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400242Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 1, Offset: 14 is stored on disk
2025-10-26T18:06:21.400264Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400301Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 2, Offset: 14 is stored on disk
2025-10-26T18:06:21.400321Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400360Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 3, Offset: 14 is stored on disk
2025-10-26T18:06:21.400402Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400442Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 4, Offset: 14 is stored on disk
2025-10-26T18:06:21.400464Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400505Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 0, Offset: 15 is stored on disk
2025-10-26T18:06:21.400526Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400568Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 1, Offset: 15 is stored on disk
2025-10-26T18:06:21.400586Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400629Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 2, Offset: 15 is stored on disk
2025-10-26T18:06:21.400649Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400692Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 3, Offset: 15 is stored on disk
2025-10-26T18:06:21.400724Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400761Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 4, Offset: 15 is stored on disk
2025-10-26T18:06:21.400782Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400818Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 0, Offset: 16 is stored on disk
2025-10-26T18:06:21.400834Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400879Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 1, Offset: 16 is stored on disk
2025-10-26T18:06:21.400897Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.400940Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 2, Offset: 16 is stored on disk
2025-10-26T18:06:21.400971Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.401024Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 3, Offset: 16 is stored on disk
2025-10-26T18:06:21.401064Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.401116Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 4, Offset: 16 is stored on disk
2025-10-26T18:06:21.410033Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 29  requestId: <none> cookie: 15
2025-10-26T18:06:21.410241Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 31  requestId: <none> cookie: 16
2025-10-26T18:06:21.410391Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 33  requestId: <none> cookie: 17
2025-10-26T18:06:21.411026Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.411125Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.411160Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.415096Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 15 offsets: 14 already_written: false write_statistics { persist_duration_ms: 389 queued_in_partition_duration_ms: 282 }
2025-10-26T18:06:21.415253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 15
2025-10-26T18:06:21.415285Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:21.417067Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 16 offsets: 15 already_written: false write_statistics { persist_duration_ms: 304 queued_in_partition_duration_ms: 293 }
2025-10-26T18:06:21.417116Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 16
2025-10-26T18:06:21.418036Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 17 offsets: 16 already_written: false write_statistics { persist_duration_ms: 289 queued_in_partition_duration_ms: 252 }
2025-10-26T18:06:21.418069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 17
2025-10-26T18:06:21.421764Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.421824Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.422487Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 0 messageNo: 47 size: 511959
2025-10-26T18:06:21.423537Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 1 messageNo: 47 size: 511959
2025-10-26T18:06:21.425620Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 2 messageNo: 47 size: 511959
2025-10-26T18:06:21.426278Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 3 messageNo: 47 size: 511959
2025-10-26T18:06:21.426537Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 4 messageNo: 47 size: 205040
2025-10-26T18:06:21.426560Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 4 messageNo: 47 size 205040 offset: -1
2025-10-26T18:06:21.431141Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 0
2025-10-26T18:06:21.433509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 25 to 25
2025-10-26T18:06:21.435528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:21.435571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 25
2025-10-26T18:06:21.436799Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 1
2025-10-26T18:06:21.436838Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 2
2025-10-26T18:06:21.436859Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 3
2025-10-26T18:06:21.436876Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 4
2025-10-26T18:06:21.442358Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (3-3)
2025-10-26T18:06:21.442425Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 3 ProducerId: "src_id" SeqNo: 4 CreateTime: 2025-10-26T18:06:18.533000Z WriteTime: 2025-10-26T18:06:19.157000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:21.446359Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (4-4)
2025-10-26T18:06:21.454483Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:21.455453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 26 to 26
2025-10-26T18:06:21.457963Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:21.457995Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 26
2025-10-26T18:06:21.465802Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:21.471190Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 18 partNo 4 FormedBlobsCount 0 NewHead: Offset 17 PartNo 0 PackedSize 2253230 count 1 nextOffset 18 batches 5
2025-10-26T18:06:21.471477Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 0
2025-10-26T18:06:21.474977Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:21.475808Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.475878Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.476273Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:21.489486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 27 to 27
2025-10-26T18:06:21.492043Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:21.492094Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 27
2025-10-26T18:06:21.496453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:21.497364Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (4-4)
2025-10-26T18:06:21.497420Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 4 ProducerId: "src_id" SeqNo: 5 CreateTime: 2025-10-26T18:06:18.568000Z WriteTime: 2025-10-26T18:06:19.509000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:50340", "_ip": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:21.497763Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (5-5)
2025-10-26T18:06:21.497825Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:21.503980Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 19 partNo 0 result is x0000000000_00000000000000000014_00002_0000000004_00014 size 7988778
2025-10-26T18:06:21.512409Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 1
2025-10-26T18:06:21.512471Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 2
2025-10-26T18:06:21.512490Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 3
2025-10-26T18:06:21.512508Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 4
2025-10-26T18:06:21.512582Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000014_00002_0000000004_00014 new key d0000000000_00000000000000000014_00002_0000000004_00014 size 7988778 WTime 1761501981512
2025-10-26T18:06:21.529208Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 19 partNo 4 FormedBlobsCount 1 NewHead: Offset 18 PartNo 0 PackedSize 2253230 count 1 nextOffset 19 batches 5
2025-10-26T18:06:21.529344Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 0
2025-10-26T18:06:21.529382Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 1
2025-10-26T18:06:21.529399Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 2
2025-10-26T18:06:21.529418Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 3
2025-10-26T18:06:21.529451Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 4
2025-10-26T18:06:21.551262Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 20 partNo 4 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 4506428 count 2 nextOffset 20 batches 9
2025-10-26T18:06:21.551419Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 0
2025-10-26T18:06:21.551460Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 1
2025-10-26T18:06:21.551490Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 2
2025-10-26T18:06:21.551506Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 3
2025-10-26T18:06:21.551526Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 4
2025-10-26T18:06:21.556041Z node 1 :FLAT_TX_SCHEMESHARD WARN: Cannot get console configs
2025-10-26T18:06:21.556081Z node 1 :IMPORT WARN: Table profiles were not loaded
2025-10-26T18:06:21.574910Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 21 partNo 4 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 6759626 count 3 nextOffset 21 batches 13
2025-10-26T18:06:21.575049Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 0
2025-10-26T18:06:21.575086Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 1
2025-10-26T18:06:21.575106Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 2
2025-10-26T18:06:21.575126Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 3
2025-10-26T18:06:21.627182Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 22 partNo 3 result is x0000000000_00000000000000000018_00000_0000000003_00015 size 8295638
2025-10-26T18:06:21.633310Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 4
2025-10-26T18:06:21.633447Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000018_00000_0000000003_00015 new key d0000000000_00000000000000000018_00000_0000000003_00015 size 8295638 WTime 1761501981633
2025-10-26T18:06:21.638156Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 22 partNo 4 FormedBlobsCount 1 NewHead: Offset 21 PartNo 3 PackedSize 717145 count 1 nextOffset 22 batches 2
2025-10-26T18:06:21.638260Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 0
2025-10-26T18:06:21.638295Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 1
2025-10-26T18:06:21.638316Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 2
2025-10-26T18:06:21.638333Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 3
2025-10-26T18:06:21.638352Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 4
2025-10-26T18:06:21.648742Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 23 partNo 4 FormedBlobsCount 0 NewHead: Offset 21 PartNo 3 PackedSize 2970343 count 2 nextOffset 23 batches 6
2025-10-26T18:06:21.649748Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 21,2 HeadOffset 14 endOffset 17 curOffset 23 d0000000000_00000000000000000021_00003_0000000002_00005| size 2970333 WTime 1761501981649
2025-10-26T18:06:21.655751Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:21.659051Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 48  requestId: <none> cookie: 25
2025-10-26T18:06:21.662270Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.680633Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.680684Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.681544Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:21.682078Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 0 messageNo: 49 size: 511959
2025-10-26T18:06:21.682613Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 1 messageNo: 49 size: 511959
2025-10-26T18:06:21.683162Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 2 messageNo: 49 size: 511959
2025-10-26T18:06:21.684781Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 3 messageNo: 49 size: 511959
2025-10-26T18:06:21.684984Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 4 messageNo: 49 size: 205040
2025-10-26T18:06:21.684999Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 4 messageNo: 49 size 205040 offset: -1
2025-10-26T18:06:21.687609Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:21.688121Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.688154Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.688254Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 50  requestId: <none> cookie: 26
2025-10-26T18:06:21.688483Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.690997Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 2 count 4 size 7988778
2025-10-26T18:06:21.698938Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:21.700642Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.700689Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.701254Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 51 size: 511959
2025-10-26T18:06:21.701844Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 51 size: 511959
2025-10-26T18:06:21.702614Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 51 size: 511959
2025-10-26T18:06:21.703109Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 51 size: 511959
2025-10-26T18:06:21.703348Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 51 size: 205040
2025-10-26T18:06:21.703371Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 51 size 205040 offset: -1
2025-10-26T18:06:21.705649Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 3 size 8295638
2025-10-26T18:06:21.710109Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:21.721878Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.721920Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.722201Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 52  requestId: <none> cookie: 27
2025-10-26T18:06:21.724335Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:21.736095Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:21.736137Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:21.736741Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 53 size: 511959
2025-10-26T18:06:21.737336Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 53 size: 511959
2025-10-26T18:06:21.737903Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 53 size: 511959
2025-10-26T18:06:21.738468Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 53 size: 511959
2025-10-26T18:06:21.738751Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 53 size: 205040
2025-10-26T18:06:21.738772Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 53 size 205040 offset: -1
2025-10-26T18:06:21.836256Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 14 count 4 size 7988778 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:21.836309Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 18 count 3 size 8295638 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:21.836451Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 13517466 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:21.836469Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836499Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 0, Offset: 17 is stored on disk
2025-10-26T18:06:21.836513Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836529Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 1, Offset: 17 is stored on disk
2025-10-26T18:06:21.836536Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836553Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 2, Offset: 17 is stored on disk
2025-10-26T18:06:21.836561Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836578Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 3, Offset: 17 is stored on disk
2025-10-26T18:06:21.836598Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836616Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 4, Offset: 17 is stored on disk
2025-10-26T18:06:21.836624Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836639Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 0, Offset: 18 is stored on disk
2025-10-26T18:06:21.836646Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836661Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 1, Offset: 18 is stored on disk
2025-10-26T18:06:21.836668Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836686Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 2, Offset: 18 is stored on disk
2025-10-26T18:06:21.836693Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836734Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 3, Offset: 18 is stored on disk
2025-10-26T18:06:21.836748Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836765Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 4, Offset: 18 is stored on disk
2025-10-26T18:06:21.836771Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836787Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 0, Offset: 19 is stored on disk
2025-10-26T18:06:21.836794Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836813Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 1, Offset: 19 is stored on disk
2025-10-26T18:06:21.836821Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836839Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 2, Offset: 19 is stored on disk
2025-10-26T18:06:21.836845Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836862Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 3, Offset: 19 is stored on disk
2025-10-26T18:06:21.836872Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836890Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 4, Offset: 19 is stored on disk
2025-10-26T18:06:21.836904Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836926Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 0, Offset: 20 is stored on disk
2025-10-26T18:06:21.836934Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.836969Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 1, Offset: 20 is stored on disk
2025-10-26T18:06:21.836978Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837005Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 2, Offset: 20 is stored on disk
2025-10-26T18:06:21.837013Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837138Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 0 size 8295679
2025-10-26T18:06:21.837160Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 size 7988778
2025-10-26T18:06:21.837186Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 3 size 7988778
2025-10-26T18:06:21.837210Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 size 8295638
2025-10-26T18:06:21.837376Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 3, Offset: 20 is stored on disk
2025-10-26T18:06:21.837394Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837416Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 4, Offset: 20 is stored on disk
2025-10-26T18:06:21.837424Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837455Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 0, Offset: 21 is stored on disk
2025-10-26T18:06:21.837462Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837477Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 1, Offset: 21 is stored on disk
2025-10-26T18:06:21.837483Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837508Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 2, Offset: 21 is stored on disk
2025-10-26T18:06:21.837523Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837547Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 3, Offset: 21 is stored on disk
2025-10-26T18:06:21.837566Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837586Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 4, Offset: 21 is stored on disk
2025-10-26T18:06:21.837595Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837638Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 0, Offset: 22 is stored on disk
2025-10-26T18:06:21.837646Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837664Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 1, Offset: 22 is stored on disk
2025-10-26T18:06:21.837672Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837703Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 2, Offset: 22 is stored on disk
2025-10-26T18:06:21.837714Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837738Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 3, Offset: 22 is stored on disk
2025-10-26T18:06:21.837749Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:21.837765Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 4, Offset: 22 is stored on disk
2025-10-26T18:06:21.852540Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 0
2025-10-26T18:06:21.852609Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 1
2025-10-26T18:06:21.852628Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 2
2025-10-26T18:06:21.852647Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 3
2025-10-26T18:06:21.852662Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 4
2025-10-26T18:06:21.908180Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 24 partNo 4 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 2253230 count 1 nextOffset 24 batches 5
2025-10-26T18:06:21.908330Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 0
2025-10-26T18:06:21.908359Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 1
2025-10-26T18:06:21.908377Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 2
2025-10-26T18:06:21.908392Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 3
2025-10-26T18:06:21.908408Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 4
2025-10-26T18:06:21.932694Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 25 partNo 4 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 4506428 count 2 nextOffset 25 batches 9
2025-10-26T18:06:21.932819Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 0
2025-10-26T18:06:21.932850Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 1
2025-10-26T18:06:22.112727Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 26 partNo 1 result is x0000000000_00000000000000000021_00003_0000000004_00014 size 7988782
2025-10-26T18:06:22.119270Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 2
2025-10-26T18:06:22.119326Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 3
2025-10-26T18:06:22.119344Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 4
2025-10-26T18:06:22.119405Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000021_00003_0000000004_00014 new key d0000000000_00000000000000000021_00003_0000000004_00014 size 7988782 WTime 1761501982118
2025-10-26T18:06:22.150645Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 topic 'rt3.dc1--test-topicCounters. CacheSize 48853212 CachedBlobs 6
2025-10-26T18:06:22.150744Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 35  requestId: <none> cookie: 18
2025-10-26T18:06:22.150829Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 37  requestId: <none> cookie: 19
2025-10-26T18:06:22.150884Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 39  requestId: <none> cookie: 20
2025-10-26T18:06:22.150939Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 41  requestId: <none> cookie: 21
2025-10-26T18:06:22.150988Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 43  requestId: <none> cookie: 22
2025-10-26T18:06:22.151042Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 45  requestId: <none> cookie: 23
2025-10-26T18:06:22.151134Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 0 size 8295679 cause it's been evicted from L2. Actual L1 size: 5
2025-10-26T18:06:22.151148Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 3 size 7988778 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:22.151162Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:22.154746Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.161782Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.161927Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.161963Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.162002Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.168240Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.216209Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 18 offsets: 17 already_written: false write_statistics { persist_duration_ms: 399 queued_in_partition_duration_ms: 283 }
2025-10-26T18:06:22.216257Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 18
2025-10-26T18:06:22.216275Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:22.255473Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 26 partNo 4 FormedBlobsCount 1 NewHead: Offset 25 PartNo 1 PackedSize 1741199 count 1 nextOffset 26 batches 4
2025-10-26T18:06:22.255599Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 0
2025-10-26T18:06:22.255627Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 1
2025-10-26T18:06:22.255640Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 2
2025-10-26T18:06:22.255654Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 3
2025-10-26T18:06:22.255666Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 4
2025-10-26T18:06:22.276878Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 19 offsets: 18 already_written: false write_statistics { persist_duration_ms: 324 queued_in_partition_duration_ms: 311 }
2025-10-26T18:06:22.276932Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 19
2025-10-26T18:06:22.281361Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 20 offsets: 19 already_written: false write_statistics { persist_duration_ms: 306 queued_in_partition_duration_ms: 306 }
2025-10-26T18:06:22.281417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 20
2025-10-26T18:06:22.282358Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 21 offsets: 20 already_written: false write_statistics { persist_duration_ms: 285 queued_in_partition_duration_ms: 228 }
2025-10-26T18:06:22.282395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 21
2025-10-26T18:06:22.283318Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 22 offsets: 21 already_written: false write_statistics { persist_duration_ms: 261 queued_in_partition_duration_ms: 229 }
2025-10-26T18:06:22.283349Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 22
2025-10-26T18:06:22.284252Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 23 offsets: 22 already_written: false write_statistics { persist_duration_ms: 197 queued_in_partition_duration_ms: 248 }
2025-10-26T18:06:22.284325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 23
2025-10-26T18:06:22.288820Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 28 to 28
2025-10-26T18:06:22.291278Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:22.291325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 28
2025-10-26T18:06:22.298105Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:22.302795Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 27 partNo 4 FormedBlobsCount 0 NewHead: Offset 25 PartNo 1 PackedSize 3994397 count 2 nextOffset 27 batches 8
2025-10-26T18:06:22.303850Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 25,2 HeadOffset 21 endOffset 23 curOffset 27 d0000000000_00000000000000000025_00001_0000000002_00007| size 3994387 WTime 1761501982303
2025-10-26T18:06:22.312803Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 29 to 29
2025-10-26T18:06:22.317892Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:22.323289Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:22.323339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 29
2025-10-26T18:06:22.330347Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:22.331169Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.331202Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.334185Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:22.334328Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 54  requestId: <none> cookie: 28
2025-10-26T18:06:22.334586Z :DEBUG: [/Root] Take Data. Partition 0. Read: {2, 0} (5-5)
2025-10-26T18:06:22.334646Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 5 ProducerId: "src_id" SeqNo: 6 CreateTime: 2025-10-26T18:06:18.595000Z WriteTime: 2025-10-26T18:06:19.550000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:50340", "_ip": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:22.334682Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.351652Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:22.343689Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (6-6)
2025-10-26T18:06:22.354475Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:22.368938Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.368995Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.369552Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 55 size: 511959
2025-10-26T18:06:22.371719Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:22.372870Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 55 size: 511959
2025-10-26T18:06:22.373459Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 55 size: 511959
2025-10-26T18:06:22.376853Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 55 size: 511959
2025-10-26T18:06:22.382223Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 55 size: 205040
2025-10-26T18:06:22.382260Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 55 size 205040 offset: -1
2025-10-26T18:06:22.383650Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.383680Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.383840Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 56  requestId: <none> cookie: 29
2025-10-26T18:06:22.386255Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.407712Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 30 to 30
2025-10-26T18:06:22.412053Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 3 count 4 size 7988782
2025-10-26T18:06:22.414347Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.414390Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.414953Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 57 size: 511959
2025-10-26T18:06:22.416216Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:22.416261Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 30
2025-10-26T18:06:22.421206Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }2025-10-26T18:06:22.422342Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 57 size: 511959
2025-10-26T18:06:22.431692Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 57 size: 511959
2025-10-26T18:06:22.436362Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 57 size: 511959
2025-10-26T18:06:22.441775Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:22.439156Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 57 size: 205040
2025-10-26T18:06:22.439200Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 57 size 205040 offset: -1
2025-10-26T18:06:22.451881Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:22.461069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 31 to 31
2025-10-26T18:06:22.463374Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:22.463418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 31
2025-10-26T18:06:22.460674Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.460717Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.460857Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 58  requestId: <none> cookie: 30
2025-10-26T18:06:22.465962Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.470549Z :DEBUG: [/Root] Take Data. Partition 0. Read: {3, 0} (6-6)
2025-10-26T18:06:22.470615Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-26T18:06:22.470644Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] In ContinueReadingDataImpl, ReadSizeBudget = 9011863, ReadSizeServerDelta = -8487575
2025-10-26T18:06:22.470878Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-26T18:06:22.470945Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 6 ProducerId: "src_id" SeqNo: 7 CreateTime: 2025-10-26T18:06:18.696000Z WriteTime: 2025-10-26T18:06:19.571000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:22.483767Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 1, data# { read_request { bytes_size: 9011863 } }
2025-10-26T18:06:22.483852Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 got read request: guid# 4500ffd2-463f6a79-adf6111c-8e975e8b
2025-10-26T18:06:22.483910Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 performing read request: guid# ff3bc2b7-53153a07-df52d80a-c34a7784, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 3, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-26T18:06:22.483975Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 3 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 7 EndOffset 10 ClientCommitOffset 0 committedOffset 0 Guid ff3bc2b7-53153a07-df52d80a-c34a7784
2025-10-26T18:06:22.487631Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:22.490459Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.490505Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.491074Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 59 size: 511959
2025-10-26T18:06:22.491530Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 59 size: 511959
2025-10-26T18:06:22.494039Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 59 size: 511959
2025-10-26T18:06:22.494692Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 59 size: 511959
2025-10-26T18:06:22.496892Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 59 size: 205040
2025-10-26T18:06:22.496931Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 59 size 205040 offset: -1
2025-10-26T18:06:22.498665Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.498689Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.498853Z node 2 :PERSQUEUE DEBUG: read cookie 6 Topic 'rt3.dc1--test-topic' partition 0 user user offset 7 count 3 size 524288 endOffset 23 max time lag 0ms effective offset 7
2025-10-26T18:06:22.499080Z node 2 :PERSQUEUE DEBUG: read cookie 6 added 2 blobs, size 8295579 count 3 last offset 10
2025-10-26T18:06:22.499149Z node 2 :PERSQUEUE DEBUG: Reading cookie 6. Send blob request.
2025-10-26T18:06:22.499200Z node 2 :PERSQUEUE DEBUG: No blob in L1. Partition 0 offset 3 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:22.499232Z node 2 :PERSQUEUE DEBUG: Got data from cache. Partition 0 offset 7 count 3 source 1 size 8295579 accessed 0 times before, last time 2025-10-26T18:06:20.000000Z
2025-10-26T18:06:22.499273Z node 2 :PERSQUEUE DEBUG: Reading cookie 6. Have to read 1 of 2 from KV
2025-10-26T18:06:22.499570Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7
2025-10-26T18:06:22.499607Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 3
2025-10-26T18:06:22.500428Z node 2 :PERSQUEUE DEBUG: Read proxy: bootstrap for direct read id: 0
2025-10-26T18:06:22.506249Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.506293Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.506383Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 60  requestId: <none> cookie: 31
2025-10-26T18:06:22.505747Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:22.508433Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.509816Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 32 to 32
2025-10-26T18:06:22.514872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:22.514927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 32
2025-10-26T18:06:22.519958Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.520010Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.520573Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 61 size: 511959
2025-10-26T18:06:22.521122Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 61 size: 511959
2025-10-26T18:06:22.522420Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 61 size: 511959
2025-10-26T18:06:22.523371Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 61 size: 511959
2025-10-26T18:06:22.523672Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 61 size: 205040
2025-10-26T18:06:22.523698Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 61 size 205040 offset: -1
2025-10-26T18:06:22.527867Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:22.537574Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 21 count 4 size 7988782 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:22.537726Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 9011644 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:22.537753Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.537790Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 0, Offset: 23 is stored on disk
2025-10-26T18:06:22.537802Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.537821Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 1, Offset: 23 is stored on disk
2025-10-26T18:06:22.537827Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.537850Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 2, Offset: 23 is stored on disk
2025-10-26T18:06:22.537859Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.537879Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 3, Offset: 23 is stored on disk
2025-10-26T18:06:22.537900Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.537921Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 4, Offset: 23 is stored on disk
2025-10-26T18:06:22.537930Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.537949Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 0, Offset: 24 is stored on disk
2025-10-26T18:06:22.537956Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.537974Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 1, Offset: 24 is stored on disk
2025-10-26T18:06:22.537983Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.538000Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 2, Offset: 24 is stored on disk
2025-10-26T18:06:22.538006Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.538026Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 3, Offset: 24 is stored on disk
2025-10-26T18:06:22.538036Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.538054Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 4, Offset: 24 is stored on disk
2025-10-26T18:06:22.538061Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.538078Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 26, partNo: 0, Offset: 25 is stored on disk
2025-10-26T18:06:22.538087Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.538104Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 26, partNo: 1, Offset: 25 is stored on disk
2025-10-26T18:06:22.538111Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.538750Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 26, partNo: 2, Offset: 25 is stored on disk
2025-10-26T18:06:22.538811Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.538882Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 26, partNo: 3, Offset: 25 is stored on disk
2025-10-26T18:06:22.538919Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.538972Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 26, partNo: 4, Offset: 25 is stored on disk
2025-10-26T18:06:22.538992Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.539048Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 27, partNo: 0, Offset: 26 is stored on disk
2025-10-26T18:06:22.539067Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.539116Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 27, partNo: 1, Offset: 26 is stored on disk
2025-10-26T18:06:22.539145Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.539209Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 27, partNo: 2, Offset: 26 is stored on disk
2025-10-26T18:06:22.539223Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.539275Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 27, partNo: 3, Offset: 26 is stored on disk
2025-10-26T18:06:22.539316Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.539363Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 27, partNo: 4, Offset: 26 is stored on disk
2025-10-26T18:06:22.544988Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:22.546207Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 10 size 7988760
2025-10-26T18:06:22.546352Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 size 7988782
2025-10-26T18:06:22.546658Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 47  requestId: <none> cookie: 24
2025-10-26T18:06:22.546720Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 49  requestId: <none> cookie: 25
2025-10-26T18:06:22.546772Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 51  requestId: <none> cookie: 26
2025-10-26T18:06:22.546818Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 53  requestId: <none> cookie: 27
2025-10-26T18:06:22.546860Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.546875Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.546921Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 10 size 7988760 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:22.546930Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:22.548386Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.548483Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.548523Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.548561Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.581743Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:22.583445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 24 offsets: 23 already_written: false write_statistics { persist_duration_ms: 686 queued_in_partition_duration_ms: 196 }
2025-10-26T18:06:22.583498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 24
2025-10-26T18:06:22.585137Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 25 offsets: 24 already_written: false write_statistics { persist_duration_ms: 630 queued_in_partition_duration_ms: 220 }
2025-10-26T18:06:22.585169Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 25
2025-10-26T18:06:22.586108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 26 offsets: 25 already_written: false write_statistics { persist_duration_ms: 605 queued_in_partition_duration_ms: 227 }
2025-10-26T18:06:22.586141Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 26
2025-10-26T18:06:22.587010Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 27 offsets: 26 already_written: false write_statistics { persist_duration_ms: 283 queued_in_partition_duration_ms: 515 }
2025-10-26T18:06:22.587039Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 27
2025-10-26T18:06:22.595015Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 0
2025-10-26T18:06:22.595072Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 1
2025-10-26T18:06:22.595094Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 2
2025-10-26T18:06:22.595113Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 3
2025-10-26T18:06:22.595130Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 4
2025-10-26T18:06:22.605601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 33 to 33
2025-10-26T18:06:22.623760Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:22.623801Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 33
2025-10-26T18:06:22.634016Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:22.657239Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 34 to 34
2025-10-26T18:06:22.659628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:22.659673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 34
2025-10-26T18:06:22.662756Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 28 partNo 4 FormedBlobsCount 0 NewHead: Offset 27 PartNo 0 PackedSize 2253230 count 1 nextOffset 28 batches 5
2025-10-26T18:06:22.662906Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 0
2025-10-26T18:06:22.662952Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 1
2025-10-26T18:06:22.662978Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 2
2025-10-26T18:06:22.662998Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 3
2025-10-26T18:06:22.663018Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 4
2025-10-26T18:06:22.669529Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:22.680347Z node 2 :PERSQUEUE DEBUG: Got results. 1 of 2 from KV. Status 1
2025-10-26T18:06:22.680384Z node 2 :PERSQUEUE DEBUG: Got results. result 0 from KV. Status 0
2025-10-26T18:06:22.680433Z node 2 :PERSQUEUE DEBUG: Prefetched blob in L1. Partition 0 offset 3 count 4 size 7988778 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:22.680556Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 14 size 7988778
2025-10-26T18:06:22.680586Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 size 7988778
2025-10-26T18:06:22.692140Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 35 to 35
2025-10-26T18:06:22.701928Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 14 size 7988778 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:22.701970Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:22.703722Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:22.703756Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 35
2025-10-26T18:06:22.710761Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:22.722927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 36 to 36
2025-10-26T18:06:22.725105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:22.725158Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 36
2025-10-26T18:06:22.726328Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:22.725612Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 29 partNo 4 result is x0000000000_00000000000000000025_00001_0000000003_00015 size 8295604
2025-10-26T18:06:22.725744Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000025_00001_0000000003_00015 new key d0000000000_00000000000000000025_00001_0000000003_00015 size 8295604 WTime 1761501982725
2025-10-26T18:06:22.728389Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 29 partNo 4 FormedBlobsCount 1 NewHead: Offset 28 PartNo 4 PackedSize 205118 count 1 nextOffset 29 batches 1
2025-10-26T18:06:22.736649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 37 to 37
2025-10-26T18:06:22.739743Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 0
2025-10-26T18:06:22.739813Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 1
2025-10-26T18:06:22.739833Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 2
2025-10-26T18:06:22.739851Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 3
2025-10-26T18:06:22.739868Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 4
2025-10-26T18:06:22.743672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:22.743721Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 37
2025-10-26T18:06:22.743755Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:22.744253Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:22.767853Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 30 partNo 4 FormedBlobsCount 0 NewHead: Offset 28 PartNo 4 PackedSize 2458316 count 2 nextOffset 30 batches 5
2025-10-26T18:06:22.767998Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 0
2025-10-26T18:06:22.768035Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 1
2025-10-26T18:06:22.768057Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 2
2025-10-26T18:06:22.768078Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 3
2025-10-26T18:06:22.768099Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 4
2025-10-26T18:06:22.783495Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 31 partNo 4 FormedBlobsCount 0 NewHead: Offset 28 PartNo 4 PackedSize 4711514 count 3 nextOffset 31 batches 9
2025-10-26T18:06:22.784512Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 28,3 HeadOffset 25 endOffset 27 curOffset 31 d0000000000_00000000000000000028_00004_0000000003_00008| size 4711504 WTime 1761501982783
2025-10-26T18:06:22.794048Z node 2 :PERSQUEUE DEBUG: FormAnswer 2
2025-10-26T18:06:22.797758Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:22.826795Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 1 count 3 size 8295604
2025-10-26T18:06:22.834043Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 7 totakecount 4 count 0 size 512009 from pos 0 cbcount 1
2025-10-26T18:06:22.837547Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 7 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:22.838068Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 62  requestId: <none> cookie: 32
2025-10-26T18:06:22.839471Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 7 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:22.840794Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 7 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:22.842602Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 7 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-26T18:06:22.843541Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.844427Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 8 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:22.846024Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 8 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:22.847732Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 8 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:22.852839Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 8 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-26T18:06:22.860312Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 9 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:22.862539Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:22.873257Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 9 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:22.879244Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 9 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:22.880324Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 9 totakecount 3 count 1 size 205085 from pos 0 cbcount 1
2025-10-26T18:06:22.887224Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:22.887524Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.887568Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.888089Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 63 size: 511959
2025-10-26T18:06:22.888589Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 63 size: 511959
2025-10-26T18:06:22.890193Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 63 size: 511959
2025-10-26T18:06:22.891366Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 63 size: 511959
2025-10-26T18:06:22.891685Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 63 size: 205040
2025-10-26T18:06:22.891715Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 63 size 205040 offset: -1
2025-10-26T18:06:22.900320Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 7
2025-10-26T18:06:22.900669Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.900695Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.905973Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 64  requestId: <none> cookie: 33
2025-10-26T18:06:22.907577Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.916740Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:22.942349Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:22.942758Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.942800Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.944173Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 65 size: 511959
2025-10-26T18:06:22.944706Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 65 size: 511959
2025-10-26T18:06:22.945227Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 65 size: 511959
2025-10-26T18:06:22.948509Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 65 size: 511959
2025-10-26T18:06:22.949023Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 65 size: 205040
2025-10-26T18:06:22.949057Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 65 size 205040 offset: -1
2025-10-26T18:06:22.954462Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.954526Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.955400Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 66  requestId: <none> cookie: 34
2025-10-26T18:06:22.962734Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:22.978492Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:22.981074Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:22.981146Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:22.983957Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 67 size: 511959
2025-10-26T18:06:22.987594Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 27 Result { Offset: 7 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 8 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1761501979865 CreateTimestampMS: 1761501978772 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 8 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 9 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1761501979926 CreateTimestampMS: 1761501978927 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 9 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 10 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1761501979940 CreateTimestampMS: 1761501978976 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 1 BlobsFromCache: 1 SizeLag: 45781371 RealReadOffset: 9 WaitQuotaTimeMs: 0 } Cookie: 7 }
2025-10-26T18:06:22.987878Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 67 size: 511959
2025-10-26T18:06:22.988966Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 67 size: 511959
2025-10-26T18:06:22.989874Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 67 size: 511959
2025-10-26T18:06:22.990117Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 67 size: 205040
2025-10-26T18:06:22.990138Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 67 size 205040 offset: -1
2025-10-26T18:06:22.992909Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 25 count 3 size 8295604 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:22.993045Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 18 size 8295638
2025-10-26T18:06:22.993072Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 size 8295604
2025-10-26T18:06:22.993094Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 9011644 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:22.993120Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993161Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 28, partNo: 0, Offset: 27 is stored on disk
2025-10-26T18:06:22.993180Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993203Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 28, partNo: 1, Offset: 27 is stored on disk
2025-10-26T18:06:22.993210Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993232Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 28, partNo: 2, Offset: 27 is stored on disk
2025-10-26T18:06:22.993242Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993288Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 28, partNo: 3, Offset: 27 is stored on disk
2025-10-26T18:06:22.993341Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993385Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 28, partNo: 4, Offset: 27 is stored on disk
2025-10-26T18:06:22.993403Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993471Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 29, partNo: 0, Offset: 28 is stored on disk
2025-10-26T18:06:22.993484Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993508Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 29, partNo: 1, Offset: 28 is stored on disk
2025-10-26T18:06:22.993518Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993537Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 29, partNo: 2, Offset: 28 is stored on disk
2025-10-26T18:06:22.993545Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993563Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 29, partNo: 3, Offset: 28 is stored on disk
2025-10-26T18:06:22.993575Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993593Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 29, partNo: 4, Offset: 28 is stored on disk
2025-10-26T18:06:22.993601Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993620Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 30, partNo: 0, Offset: 29 is stored on disk
2025-10-26T18:06:22.993630Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993647Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 30, partNo: 1, Offset: 29 is stored on disk
2025-10-26T18:06:22.993654Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993673Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 30, partNo: 2, Offset: 29 is stored on disk
2025-10-26T18:06:22.993703Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993721Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 30, partNo: 3, Offset: 29 is stored on disk
2025-10-26T18:06:22.993732Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993751Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 30, partNo: 4, Offset: 29 is stored on disk
2025-10-26T18:06:22.993771Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993789Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 31, partNo: 0, Offset: 30 is stored on disk
2025-10-26T18:06:22.993796Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993816Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 31, partNo: 1, Offset: 30 is stored on disk
2025-10-26T18:06:22.993825Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993844Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 31, partNo: 2, Offset: 30 is stored on disk
2025-10-26T18:06:22.993851Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993869Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 31, partNo: 3, Offset: 30 is stored on disk
2025-10-26T18:06:22.993880Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:22.993900Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 31, partNo: 4, Offset: 30 is stored on disk
2025-10-26T18:06:22.997183Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 55  requestId: <none> cookie: 28
2025-10-26T18:06:22.997283Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 57  requestId: <none> cookie: 29
2025-10-26T18:06:22.997338Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 59  requestId: <none> cookie: 30
2025-10-26T18:06:22.997402Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 61  requestId: <none> cookie: 31
2025-10-26T18:06:22.997465Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 18 size 8295638 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:22.997486Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:23.003634Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.003722Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.003759Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.003790Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.003887Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:23.007592Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.007641Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.009048Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 28 offsets: 27 already_written: false write_statistics { persist_duration_ms: 397 queued_in_partition_duration_ms: 211 }
2025-10-26T18:06:23.009091Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 28
2025-10-26T18:06:23.009123Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:23.015752Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 29 offsets: 28 already_written: false write_statistics { persist_duration_ms: 330 queued_in_partition_duration_ms: 216 }
2025-10-26T18:06:23.015803Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 29
2025-10-26T18:06:23.024693Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 38 to 38
2025-10-26T18:06:23.033932Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 0
2025-10-26T18:06:23.033998Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 1
2025-10-26T18:06:23.034019Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 2
2025-10-26T18:06:23.034040Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 3
2025-10-26T18:06:23.034057Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 4
2025-10-26T18:06:23.035505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:23.035549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 38
2025-10-26T18:06:23.036114Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:23.041875Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 30 offsets: 29 already_written: false write_statistics { persist_duration_ms: 253 queued_in_partition_duration_ms: 241 }
2025-10-26T18:06:23.041924Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 30
2025-10-26T18:06:23.049411Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 31 offsets: 30 already_written: false write_statistics { persist_duration_ms: 225 queued_in_partition_duration_ms: 241 }
2025-10-26T18:06:23.049474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 31
2025-10-26T18:06:23.063847Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 39 to 39
2025-10-26T18:06:23.071276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:23.071338Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 39
2025-10-26T18:06:23.072627Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:23.073957Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 10 endOffset 27
2025-10-26T18:06:23.074016Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 27 ReadOffset 10 ReadGuid ff3bc2b7-53153a07-df52d80a-c34a7784 has messages 1
2025-10-26T18:06:23.075830Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 10, endOffset# 27, WTime# 1761501979940, sizeLag# 45781371
2025-10-26T18:06:23.075845Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1TEvPartitionReady. Aval parts: 0
2025-10-26T18:06:23.075887Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 read done: guid# ff3bc2b7-53153a07-df52d80a-c34a7784, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 6758902
2025-10-26T18:06:23.075911Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 response to read: guid# ff3bc2b7-53153a07-df52d80a-c34a7784
2025-10-26T18:06:23.093432Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Process answer. Aval parts: 1
2025-10-26T18:06:23.095918Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 40 to 40
2025-10-26T18:06:23.099158Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:23.099202Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 40
2025-10-26T18:06:23.099807Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:23.134366Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 32 partNo 4 FormedBlobsCount 0 NewHead: Offset 31 PartNo 0 PackedSize 2253230 count 1 nextOffset 32 batches 5
2025-10-26T18:06:23.134507Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 0
2025-10-26T18:06:23.134544Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 1
2025-10-26T18:06:23.134568Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 2
2025-10-26T18:06:23.145555Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Got ReadResponse, serverBytesSize = 6758902, now ReadSizeBudget = 0, ReadSizeServerDelta = -6234614
2025-10-26T18:06:23.147220Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 41 to 41
2025-10-26T18:06:23.154512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:23.154560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 41
2025-10-26T18:06:23.154590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:23.154697Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (7-7)
2025-10-26T18:06:23.155028Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 6758902 to budget
2025-10-26T18:06:23.158919Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (7-7)
2025-10-26T18:06:23.158986Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 7 ProducerId: "src_id" SeqNo: 8 CreateTime: 2025-10-26T18:06:18.772000Z WriteTime: 2025-10-26T18:06:19.865000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:50340", "_ip": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:23.159957Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (8-8)
2025-10-26T18:06:23.159991Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:23.207665Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 33 partNo 2 result is x0000000000_00000000000000000028_00004_0000000004_00014 size 7988760
2025-10-26T18:06:23.209146Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 3
2025-10-26T18:06:23.209482Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 4
2025-10-26T18:06:23.209555Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000028_00004_0000000004_00014 new key d0000000000_00000000000000000028_00004_0000000004_00014 size 7988760 WTime 1761501983209
2025-10-26T18:06:23.224340Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 33 partNo 4 FormedBlobsCount 1 NewHead: Offset 32 PartNo 2 PackedSize 1229172 count 1 nextOffset 33 batches 3
2025-10-26T18:06:23.224462Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 0
2025-10-26T18:06:23.224500Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 1
2025-10-26T18:06:23.224518Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 2
2025-10-26T18:06:23.224537Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 3
2025-10-26T18:06:23.224555Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 4
2025-10-26T18:06:23.243430Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 34 partNo 4 FormedBlobsCount 0 NewHead: Offset 32 PartNo 2 PackedSize 3482370 count 2 nextOffset 34 batches 7
2025-10-26T18:06:23.244302Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 32,2 HeadOffset 28 endOffset 31 curOffset 34 d0000000000_00000000000000000032_00002_0000000002_00006| size 3482360 WTime 1761501983244
2025-10-26T18:06:23.252196Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:23.252276Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 68  requestId: <none> cookie: 35
2025-10-26T18:06:23.255036Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.265387Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:23.278946Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 4 count 4 size 7988760
2025-10-26T18:06:23.297006Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.297054Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.297624Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 69 size: 511959
2025-10-26T18:06:23.298148Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 69 size: 511959
2025-10-26T18:06:23.298661Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 69 size: 511959
2025-10-26T18:06:23.302126Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 69 size: 511959
2025-10-26T18:06:23.302374Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 69 size: 205040
2025-10-26T18:06:23.302395Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 69 size 205040 offset: -1
2025-10-26T18:06:23.317329Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:23.319904Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.319949Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.320039Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 70  requestId: <none> cookie: 36
2025-10-26T18:06:23.325137Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.334813Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:23.342147Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.342197Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.342749Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 36 partNo : 0 messageNo: 71 size: 511959
2025-10-26T18:06:23.357844Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 36 partNo : 1 messageNo: 71 size: 511959
2025-10-26T18:06:23.363446Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 36 partNo : 2 messageNo: 71 size: 511959
2025-10-26T18:06:23.369823Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 36 partNo : 3 messageNo: 71 size: 511959
2025-10-26T18:06:23.370103Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 36 partNo : 4 messageNo: 71 size: 205040
2025-10-26T18:06:23.370122Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 36 partNo : 4 messageNo: 71 size 205040 offset: -1
2025-10-26T18:06:23.378947Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:23.383131Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.383183Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.383274Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 72  requestId: <none> cookie: 37
2025-10-26T18:06:23.383803Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.389483Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 28 count 4 size 7988760 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:23.391069Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 6758733 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:23.391103Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391137Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 32, partNo: 0, Offset: 31 is stored on disk
2025-10-26T18:06:23.391160Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391178Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 32, partNo: 1, Offset: 31 is stored on disk
2025-10-26T18:06:23.391185Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391202Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 32, partNo: 2, Offset: 31 is stored on disk
2025-10-26T18:06:23.391219Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391240Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 32, partNo: 3, Offset: 31 is stored on disk
2025-10-26T18:06:23.391260Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391280Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 32, partNo: 4, Offset: 31 is stored on disk
2025-10-26T18:06:23.391289Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391306Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 33, partNo: 0, Offset: 32 is stored on disk
2025-10-26T18:06:23.391321Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391338Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 33, partNo: 1, Offset: 32 is stored on disk
2025-10-26T18:06:23.391345Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391362Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 33, partNo: 2, Offset: 32 is stored on disk
2025-10-26T18:06:23.391370Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391390Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 33, partNo: 3, Offset: 32 is stored on disk
2025-10-26T18:06:23.391400Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391416Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 33, partNo: 4, Offset: 32 is stored on disk
2025-10-26T18:06:23.391423Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391442Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 34, partNo: 0, Offset: 33 is stored on disk
2025-10-26T18:06:23.391450Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391469Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 34, partNo: 1, Offset: 33 is stored on disk
2025-10-26T18:06:23.391476Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391493Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 34, partNo: 2, Offset: 33 is stored on disk
2025-10-26T18:06:23.391500Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391518Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 34, partNo: 3, Offset: 33 is stored on disk
2025-10-26T18:06:23.391528Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.391546Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 34, partNo: 4, Offset: 33 is stored on disk
2025-10-26T18:06:23.399126Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 7 size 8295579
2025-10-26T18:06:23.399163Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 28 size 7988760
2025-10-26T18:06:23.405518Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 0
2025-10-26T18:06:23.405573Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 1
2025-10-26T18:06:23.405590Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 2
2025-10-26T18:06:23.405608Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 3
2025-10-26T18:06:23.405627Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 4
2025-10-26T18:06:23.410893Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:23.418804Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:23.433615Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 35 partNo 4 FormedBlobsCount 0 NewHead: Offset 34 PartNo 0 PackedSize 2253230 count 1 nextOffset 35 batches 5
2025-10-26T18:06:23.433766Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 36 partNo 0
2025-10-26T18:06:23.433808Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 36 partNo 1
2025-10-26T18:06:23.433825Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 36 partNo 2
2025-10-26T18:06:23.433840Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 36 partNo 3
2025-10-26T18:06:23.433853Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 36 partNo 4
2025-10-26T18:06:23.444051Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 36 partNo 4 FormedBlobsCount 0 NewHead: Offset 34 PartNo 0 PackedSize 4506428 count 2 nextOffset 36 batches 9
2025-10-26T18:06:23.444913Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 34,2 HeadOffset 32 endOffset 34 curOffset 36 d0000000000_00000000000000000034_00000_0000000002_00008| size 4506418 WTime 1761501983444
2025-10-26T18:06:23.455188Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 63  requestId: <none> cookie: 32
2025-10-26T18:06:23.455301Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 65  requestId: <none> cookie: 33
2025-10-26T18:06:23.455356Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 67  requestId: <none> cookie: 34
2025-10-26T18:06:23.458541Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.458585Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.459057Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 37 partNo : 0 messageNo: 73 size: 511959
2025-10-26T18:06:23.459507Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 37 partNo : 1 messageNo: 73 size: 511959
2025-10-26T18:06:23.459959Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 37 partNo : 2 messageNo: 73 size: 511959
2025-10-26T18:06:23.460412Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 37 partNo : 3 messageNo: 73 size: 511959
2025-10-26T18:06:23.460626Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 37 partNo : 4 messageNo: 73 size: 205040
2025-10-26T18:06:23.460647Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 37 partNo : 4 messageNo: 73 size 205040 offset: -1
2025-10-26T18:06:23.461787Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.461908Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.461970Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.462690Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.462737Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.462836Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 7 size 8295579 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:23.462859Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:23.467574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 32 offsets: 31 already_written: false write_statistics { persist_duration_ms: 356 queued_in_partition_duration_ms: 142 }
2025-10-26T18:06:23.467646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 32
2025-10-26T18:06:23.467670Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:23.469894Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 33 offsets: 32 already_written: false write_statistics { persist_duration_ms: 256 queued_in_partition_duration_ms: 180 }
2025-10-26T18:06:23.469437Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:23.470054Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 33
2025-10-26T18:06:23.471404Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 34 offsets: 33 already_written: false write_statistics { persist_duration_ms: 165 queued_in_partition_duration_ms: 232 }
2025-10-26T18:06:23.471435Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 34
2025-10-26T18:06:23.475150Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 42 to 42
2025-10-26T18:06:23.477602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:23.477664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 42
2025-10-26T18:06:23.479312Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:23.484473Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 74  requestId: <none> cookie: 38
2025-10-26T18:06:23.487310Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.495254Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:23.496955Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 43 to 43
2025-10-26T18:06:23.502285Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:23.502335Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 43
2025-10-26T18:06:23.506507Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.506583Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.509710Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (8-8)
2025-10-26T18:06:23.509779Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 8 ProducerId: "src_id" SeqNo: 9 CreateTime: 2025-10-26T18:06:18.927000Z WriteTime: 2025-10-26T18:06:19.926000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:50340", "_ip": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:23.510173Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (9-9)
2025-10-26T18:06:23.510247Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:23.510567Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 38 partNo : 0 messageNo: 75 size: 511959
2025-10-26T18:06:23.511510Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 38 partNo : 1 messageNo: 75 size: 511959
2025-10-26T18:06:23.512064Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 38 partNo : 2 messageNo: 75 size: 511959
2025-10-26T18:06:23.512557Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 38 partNo : 3 messageNo: 75 size: 511959
2025-10-26T18:06:23.512790Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 38 partNo : 4 messageNo: 75 size: 205040
2025-10-26T18:06:23.512809Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 38 partNo : 4 messageNo: 75 size 205040 offset: -1
2025-10-26T18:06:23.523765Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505822 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:23.523805Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.523845Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 35, partNo: 0, Offset: 34 is stored on disk
2025-10-26T18:06:23.523858Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.523877Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 35, partNo: 1, Offset: 34 is stored on disk
2025-10-26T18:06:23.523885Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.523903Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 35, partNo: 2, Offset: 34 is stored on disk
2025-10-26T18:06:23.523911Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.523930Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 35, partNo: 3, Offset: 34 is stored on disk
2025-10-26T18:06:23.523955Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.523974Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 35, partNo: 4, Offset: 34 is stored on disk
2025-10-26T18:06:23.523981Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.524000Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 36, partNo: 0, Offset: 35 is stored on disk
2025-10-26T18:06:23.524008Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.524025Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 36, partNo: 1, Offset: 35 is stored on disk
2025-10-26T18:06:23.524031Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.524049Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 36, partNo: 2, Offset: 35 is stored on disk
2025-10-26T18:06:23.524057Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.524073Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 36, partNo: 3, Offset: 35 is stored on disk
2025-10-26T18:06:23.524082Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:23.524102Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 36, partNo: 4, Offset: 35 is stored on disk
2025-10-26T18:06:23.532167Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:23.554590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 44 to 44
2025-10-26T18:06:23.564387Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 37 partNo 0
2025-10-26T18:06:23.571167Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:23.571219Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 44
2025-10-26T18:06:23.571248Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:23.572136Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:23.585319Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 69  requestId: <none> cookie: 35
2025-10-26T18:06:23.585469Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 71  requestId: <none> cookie: 36
2025-10-26T18:06:23.585564Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.585588Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.586128Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.586217Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.587888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 35 offsets: 34 already_written: false write_statistics { persist_duration_ms: 120 queued_in_partition_duration_ms: 97 }
2025-10-26T18:06:23.587977Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 35
2025-10-26T18:06:23.588031Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:23.589340Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 37 partNo 0 result is x0000000000_00000000000000000032_00002_0000000004_00014 size 7988778
2025-10-26T18:06:23.594508Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 37 partNo 1
2025-10-26T18:06:23.594558Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 37 partNo 2
2025-10-26T18:06:23.594592Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 37 partNo 3
2025-10-26T18:06:23.594631Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 37 partNo 4
2025-10-26T18:06:23.594699Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000032_00002_0000000004_00014 new key d0000000000_00000000000000000032_00002_0000000004_00014 size 7988778 WTime 1761501983594
2025-10-26T18:06:23.613205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 36 offsets: 35 already_written: false write_statistics { persist_duration_ms: 90 queued_in_partition_duration_ms: 62 }
2025-10-26T18:06:23.613252Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 36
2025-10-26T18:06:23.623951Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 45 to 45
2025-10-26T18:06:23.626058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:23.626101Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 45
2025-10-26T18:06:23.626636Z :DEBUG: [/Root] Take Data. Partition 0. Read: {2, 0} (9-9)
2025-10-26T18:06:23.626685Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-26T18:06:23.626720Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] In ContinueReadingDataImpl, ReadSizeBudget = 6758902, ReadSizeServerDelta = -6234614
2025-10-26T18:06:23.626969Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-26T18:06:23.627008Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 9 ProducerId: "src_id" SeqNo: 10 CreateTime: 2025-10-26T18:06:18.976000Z WriteTime: 2025-10-26T18:06:19.940000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:23.634137Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 1, data# { read_request { bytes_size: 6758902 } }
2025-10-26T18:06:23.634263Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 got read request: guid# 47e2b210-e15e489d-69c28810-54aef298
2025-10-26T18:06:23.634315Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 performing read request: guid# 89752e48-985abb8c-b36886e3-7f6bf24d, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 20, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-26T18:06:23.634379Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 20 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 10 EndOffset 27 ClientCommitOffset 0 committedOffset 0 Guid 89752e48-985abb8c-b36886e3-7f6bf24d
2025-10-26T18:06:23.638223Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.638272Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.638341Z node 2 :PERSQUEUE DEBUG: Read proxy: bootstrap for direct read id: 0
2025-10-26T18:06:23.645208Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 46 to 46
2025-10-26T18:06:23.647188Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:23.647228Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 46
2025-10-26T18:06:23.647255Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:23.648123Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:23.665928Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 37 partNo 4 FormedBlobsCount 1 NewHead: Offset 36 PartNo 0 PackedSize 2253230 count 1 nextOffset 37 batches 5
2025-10-26T18:06:23.666075Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 38 partNo 0
2025-10-26T18:06:23.666111Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 38 partNo 1
2025-10-26T18:06:23.666128Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 38 partNo 2
2025-10-26T18:06:23.666147Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 38 partNo 3
2025-10-26T18:06:23.666162Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 38 partNo 4
2025-10-26T18:06:23.692979Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 38 partNo 4 FormedBlobsCount 0 NewHead: Offset 36 PartNo 0 PackedSize 4506428 count 2 nextOffset 38 batches 9
2025-10-26T18:06:23.694025Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 36,2 HeadOffset 32 endOffset 36 curOffset 38 d0000000000_00000000000000000036_00000_0000000002_00008| size 4506418 WTime 1761501983693
2025-10-26T18:06:23.709591Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:23.709746Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 76  requestId: <none> cookie: 39
2025-10-26T18:06:23.709853Z node 2 :PERSQUEUE DEBUG: read cookie 7 Topic 'rt3.dc1--test-topic' partition 0 user user offset 10 count 20 size 524288 endOffset 36 max time lag 0ms effective offset 10
2025-10-26T18:06:23.710082Z node 2 :PERSQUEUE DEBUG: read cookie 7 added 2 blobs, size 7988760 count 4 last offset 14
2025-10-26T18:06:23.710167Z node 2 :PERSQUEUE DEBUG: Reading cookie 7. Send blob request.
2025-10-26T18:06:23.710613Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.720776Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.720827Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.721359Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 39 partNo : 0 messageNo: 77 size: 511959
2025-10-26T18:06:23.727670Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 39 partNo : 1 messageNo: 77 size: 511959
2025-10-26T18:06:23.728295Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 39 partNo : 2 messageNo: 77 size: 511959
2025-10-26T18:06:23.729512Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:23.734260Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 39 partNo : 3 messageNo: 77 size: 511959
2025-10-26T18:06:23.739734Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:23.737980Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 39 partNo : 4 messageNo: 77 size: 205040
2025-10-26T18:06:23.738018Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 39 partNo : 4 messageNo: 77 size 205040 offset: -1
2025-10-26T18:06:23.745846Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.745887Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.745986Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 78  requestId: <none> cookie: 40
2025-10-26T18:06:23.746382Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.748203Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 32 partNo 2 count 4 size 7988778
2025-10-26T18:06:23.760340Z node 2 :PERSQUEUE DEBUG: No blob in L1. Partition 0 offset 7 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:23.765043Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.765090Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.765614Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 40 partNo : 0 messageNo: 79 size: 511959
2025-10-26T18:06:23.767691Z node 2 :PERSQUEUE DEBUG: No blob in L1. Partition 0 offset 10 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:23.767752Z node 2 :PERSQUEUE DEBUG: Reading cookie 7. Have to read 2 of 2 from KV
2025-10-26T18:06:23.769780Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 7
2025-10-26T18:06:23.769617Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:23.770297Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 40 partNo : 1 messageNo: 79 size: 511959
2025-10-26T18:06:23.770786Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 40 partNo : 2 messageNo: 79 size: 511959
2025-10-26T18:06:23.771263Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 40 partNo : 3 messageNo: 79 size: 511959
2025-10-26T18:06:23.771480Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 40 partNo : 4 messageNo: 79 size: 205040
2025-10-26T18:06:23.771502Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 40 partNo : 4 messageNo: 79 size 205040 offset: -1
2025-10-26T18:06:23.774863Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 10
2025-10-26T18:06:23.779898Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:23.780389Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.780436Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.780545Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 80  requestId: <none> cookie: 41
2025-10-26T18:06:23.781309Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.794510Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:23.800299Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:23.808132Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.808183Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.808799Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 41 partNo : 0 messageNo: 81 size: 511959
2025-10-26T18:06:23.809375Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 41 partNo : 1 messageNo: 81 size: 511959
2025-10-26T18:06:23.810228Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 41 partNo : 2 messageNo: 81 size: 511959
2025-10-26T18:06:23.810796Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 41 partNo : 3 messageNo: 81 size: 511959
2025-10-26T18:06:23.811055Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 41 partNo : 4 messageNo: 81 size: 205040
2025-10-26T18:06:23.811077Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 41 partNo : 4 messageNo: 81 size 205040 offset: -1
2025-10-26T18:06:23.844124Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.843432Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.843483Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.843576Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 82  requestId: <none> cookie: 42
2025-10-26T18:06:23.855844Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.855897Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.856453Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 42 partNo : 0 messageNo: 83 size: 511959
2025-10-26T18:06:23.856969Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 42 partNo : 1 messageNo: 83 size: 511959
2025-10-26T18:06:23.857516Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 42 partNo : 2 messageNo: 83 size: 511959
2025-10-26T18:06:23.858116Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 42 partNo : 3 messageNo: 83 size: 511959
2025-10-26T18:06:23.858371Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 42 partNo : 4 messageNo: 83 size: 205040
2025-10-26T18:06:23.858403Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 42 partNo : 4 messageNo: 83 size 205040 offset: -1
2025-10-26T18:06:23.877585Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:23.891543Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:23.893883Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.893932Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.894053Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 84  requestId: <none> cookie: 43
2025-10-26T18:06:23.894521Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:23.907739Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:23.907792Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:23.908323Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 43 partNo : 0 messageNo: 85 size: 511959
2025-10-26T18:06:23.908823Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 43 partNo : 1 messageNo: 85 size: 511959
2025-10-26T18:06:23.909300Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 43 partNo : 2 messageNo: 85 size: 511959
2025-10-26T18:06:23.915034Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:23.915310Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 43 partNo : 3 messageNo: 85 size: 511959
2025-10-26T18:06:23.915598Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 43 partNo : 4 messageNo: 85 size: 205040
2025-10-26T18:06:23.915621Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 43 partNo : 4 messageNo: 85 size 205040 offset: -1
2025-10-26T18:06:23.955584Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:24.061949Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.061993Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.062218Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 86  requestId: <none> cookie: 44
2025-10-26T18:06:24.067213Z node 2 :PERSQUEUE DEBUG: Got results. 2 of 2 from KV. Status 1
2025-10-26T18:06:24.067438Z node 2 :PERSQUEUE DEBUG: Got results. result 0 from KV. Status 0
2025-10-26T18:06:24.067453Z node 2 :PERSQUEUE DEBUG: Got results. result 1 from KV. Status 0
2025-10-26T18:06:24.067495Z node 2 :PERSQUEUE DEBUG: Prefetched blob in L1. Partition 0 offset 7 count 3 size 8295579 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:24.067520Z node 2 :PERSQUEUE DEBUG: Prefetched blob in L1. Partition 0 offset 10 count 4 size 7988760 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:24.067777Z node 2 :PERSQUEUE DEBUG: FormAnswer 2
2025-10-26T18:06:24.067379Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.069613Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 21 size 7988782
2025-10-26T18:06:24.073840Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 size 8295579
2025-10-26T18:06:24.073910Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 3 size 7988778
2025-10-26T18:06:24.073934Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 size 7988760
2025-10-26T18:06:24.080727Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:24.090810Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:24.095092Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.095142Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.095730Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 44 partNo : 0 messageNo: 87 size: 511959
2025-10-26T18:06:24.096299Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 44 partNo : 1 messageNo: 87 size: 511959
2025-10-26T18:06:24.096870Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 44 partNo : 2 messageNo: 87 size: 511959
2025-10-26T18:06:24.099097Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 44 partNo : 3 messageNo: 87 size: 511959
2025-10-26T18:06:24.099391Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 44 partNo : 4 messageNo: 87 size: 205040
2025-10-26T18:06:24.099419Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 44 partNo : 4 messageNo: 87 size 205040 offset: -1
2025-10-26T18:06:24.100734Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 32 count 4 size 7988778 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:24.100823Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 21 size 7988782 cause it's been evicted from L2. Actual L1 size: 6
2025-10-26T18:06:24.100841Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 3 size 7988778 cause it's been evicted from L2. Actual L1 size: 5
2025-10-26T18:06:24.100857Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:24.101115Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 25 size 8295604
2025-10-26T18:06:24.101155Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 32 size 7988778
2025-10-26T18:06:24.115373Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.115418Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.115476Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 25 size 8295604 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:24.115492Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:24.130633Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 10 totakecount 3 count 0 size 2047975 from pos 0 cbcount 4
2025-10-26T18:06:24.142470Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 10 totakecount 4 count 1 size 717095 from pos 0 cbcount 2
2025-10-26T18:06:24.144100Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 11 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:24.145527Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 11 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:24.147300Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 11 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:24.148334Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 11 totakecount 4 count 1 size 205086 from pos 0 cbcount 1
2025-10-26T18:06:24.149777Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 12 totakecount 4 count 0 size 512009 from pos 0 cbcount 1
2025-10-26T18:06:24.151593Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 12 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:24.153340Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 12 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:24.155297Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 12 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:24.157537Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 12 totakecount 4 count 1 size 717095 from pos 0 cbcount 2
2025-10-26T18:06:24.160485Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 13 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:24.162384Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 13 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:24.164161Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 13 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:24.165188Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 13 totakecount 4 count 1 size 205086 from pos 0 cbcount 1
2025-10-26T18:06:24.168454Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 14 totakecount 4 count 0 size 1024014 from pos 0 cbcount 2
2025-10-26T18:06:24.173890Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505822 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:24.173923Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.173965Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 37, partNo: 0, Offset: 36 is stored on disk
2025-10-26T18:06:24.173977Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.173996Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 37, partNo: 1, Offset: 36 is stored on disk
2025-10-26T18:06:24.174004Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.174024Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 37, partNo: 2, Offset: 36 is stored on disk
2025-10-26T18:06:24.174032Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.174051Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 37, partNo: 3, Offset: 36 is stored on disk
2025-10-26T18:06:24.174073Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.174093Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 37, partNo: 4, Offset: 36 is stored on disk
2025-10-26T18:06:24.174101Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.174120Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 38, partNo: 0, Offset: 37 is stored on disk
2025-10-26T18:06:24.174127Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.174145Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 38, partNo: 1, Offset: 37 is stored on disk
2025-10-26T18:06:24.174152Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.174169Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 38, partNo: 2, Offset: 37 is stored on disk
2025-10-26T18:06:24.174176Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.174193Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 38, partNo: 3, Offset: 37 is stored on disk
2025-10-26T18:06:24.174204Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.174220Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 38, partNo: 4, Offset: 37 is stored on disk
2025-10-26T18:06:24.191371Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 39 partNo 0
2025-10-26T18:06:24.191421Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 39 partNo 1
2025-10-26T18:06:24.191436Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 39 partNo 2
2025-10-26T18:06:24.191453Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 39 partNo 3
2025-10-26T18:06:24.191469Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 39 partNo 4
2025-10-26T18:06:24.192117Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 10
2025-10-26T18:06:24.218383Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 39 partNo 4 FormedBlobsCount 0 NewHead: Offset 38 PartNo 0 PackedSize 2253230 count 1 nextOffset 39 batches 5
2025-10-26T18:06:24.218509Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 40 partNo 0
2025-10-26T18:06:24.218541Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 40 partNo 1
2025-10-26T18:06:24.218560Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 40 partNo 2
2025-10-26T18:06:24.218578Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 40 partNo 3
2025-10-26T18:06:24.245584Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 73  requestId: <none> cookie: 37
2025-10-26T18:06:24.245742Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 75  requestId: <none> cookie: 38
2025-10-26T18:06:24.261784Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.261873Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.281926Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 37 offsets: 36 already_written: false write_statistics { persist_duration_ms: 579 queued_in_partition_duration_ms: 112 }
2025-10-26T18:06:24.282157Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 37
2025-10-26T18:06:24.282188Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:24.287105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 38 offsets: 37 already_written: false write_statistics { persist_duration_ms: 508 queued_in_partition_duration_ms: 148 }
2025-10-26T18:06:24.287154Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 38
2025-10-26T18:06:24.310615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 47 to 47
2025-10-26T18:06:24.312085Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 40 partNo 3 result is x0000000000_00000000000000000036_00000_0000000003_00015 size 8295660
2025-10-26T18:06:24.313892Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 40 partNo 4
2025-10-26T18:06:24.313983Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000036_00000_0000000003_00015 new key d0000000000_00000000000000000036_00000_0000000003_00015 size 8295660 WTime 1761501984313
2025-10-26T18:06:24.325409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:24.325471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 47
2025-10-26T18:06:24.333593Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 40 partNo 4 FormedBlobsCount 1 NewHead: Offset 39 PartNo 3 PackedSize 717145 count 1 nextOffset 40 batches 2
2025-10-26T18:06:24.333759Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 41 partNo 0
2025-10-26T18:06:24.333811Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 41 partNo 1
2025-10-26T18:06:24.333850Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 41 partNo 2
2025-10-26T18:06:24.333882Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 41 partNo 3
2025-10-26T18:06:24.333914Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 41 partNo 4
2025-10-26T18:06:24.334617Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 36 Result { Offset: 10 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 11 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501980192 CreateTimestampMS: 1761501979049 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 11 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 12 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501980307 CreateTimestampMS: 1761501979253 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 12 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 13 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501980548 CreateTimestampMS: 1761501979523 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 13 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 14 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501980582 CreateTimestampMS: 1761501979599 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 2 BlobsFromCache: 0 SizeLag: 55101044 RealReadOffset: 13 WaitQuotaTimeMs: 0 } Cookie: 10 }
2025-10-26T18:06:24.368349Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 41 partNo 4 FormedBlobsCount 0 NewHead: Offset 39 PartNo 3 PackedSize 2970343 count 2 nextOffset 41 batches 6
2025-10-26T18:06:24.368472Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 42 partNo 0
2025-10-26T18:06:24.368505Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 42 partNo 1
2025-10-26T18:06:24.368523Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 42 partNo 2
2025-10-26T18:06:24.368540Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 42 partNo 3
2025-10-26T18:06:24.368555Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 42 partNo 4
2025-10-26T18:06:24.372670Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:24.402445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 48 to 48
2025-10-26T18:06:24.404698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:24.404742Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 48
2025-10-26T18:06:24.404776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:24.408721Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:24.415874Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 14 endOffset 36
2025-10-26T18:06:24.415933Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 36 ReadOffset 14 ReadGuid 89752e48-985abb8c-b36886e3-7f6bf24d has messages 1
2025-10-26T18:06:24.418001Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 14, endOffset# 36, WTime# 1761501980582, sizeLag# 55101044
2025-10-26T18:06:24.418048Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1TEvPartitionReady. Aval parts: 0
2025-10-26T18:06:24.418131Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 read done: guid# 89752e48-985abb8c-b36886e3-7f6bf24d, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 9011865
2025-10-26T18:06:24.418177Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 response to read: guid# 89752e48-985abb8c-b36886e3-7f6bf24d
2025-10-26T18:06:24.441812Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Process answer. Aval parts: 1
2025-10-26T18:06:24.446023Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 42 partNo 4 FormedBlobsCount 0 NewHead: Offset 39 PartNo 3 PackedSize 5223541 count 3 nextOffset 42 batches 10
2025-10-26T18:06:24.446178Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 43 partNo 0
2025-10-26T18:06:24.446216Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 43 partNo 1
2025-10-26T18:06:24.446239Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 43 partNo 2
2025-10-26T18:06:24.446259Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 43 partNo 3
2025-10-26T18:06:24.446278Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 43 partNo 4
2025-10-26T18:06:24.472852Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 43 partNo 4 FormedBlobsCount 0 NewHead: Offset 39 PartNo 3 PackedSize 7476739 count 4 nextOffset 43 batches 14
2025-10-26T18:06:24.472985Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 44 partNo 0
2025-10-26T18:06:24.473028Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 44 partNo 1
2025-10-26T18:06:24.473343Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Got ReadResponse, serverBytesSize = 9011865, now ReadSizeBudget = 0, ReadSizeServerDelta = -8487577
2025-10-26T18:06:24.474345Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (10-10)
2025-10-26T18:06:24.474411Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 9011865 to budget
2025-10-26T18:06:24.539126Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 44 partNo 1 result is x0000000000_00000000000000000039_00003_0000000004_00014 size 7988760
2025-10-26T18:06:24.540785Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 44 partNo 2
2025-10-26T18:06:24.540848Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 44 partNo 3
2025-10-26T18:06:24.540868Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 44 partNo 4
2025-10-26T18:06:24.540947Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000039_00003_0000000004_00014 new key d0000000000_00000000000000000039_00003_0000000004_00014 size 7988760 WTime 1761501984540
2025-10-26T18:06:24.559917Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 44 partNo 4 FormedBlobsCount 1 NewHead: Offset 43 PartNo 1 PackedSize 1741199 count 1 nextOffset 44 batches 4
2025-10-26T18:06:24.560951Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 43,1 HeadOffset 36 endOffset 38 curOffset 44 d0000000000_00000000000000000043_00001_0000000001_00003| size 1741189 WTime 1761501984560
2025-10-26T18:06:24.564694Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:24.564778Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 88  requestId: <none> cookie: 45
2025-10-26T18:06:24.565153Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.573803Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:24.576777Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.576825Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.578093Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 45 partNo : 0 messageNo: 89 size: 511959
2025-10-26T18:06:24.578820Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 45 partNo : 1 messageNo: 89 size: 511959
2025-10-26T18:06:24.579515Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 45 partNo : 2 messageNo: 89 size: 511959
2025-10-26T18:06:24.580103Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 45 partNo : 3 messageNo: 89 size: 511959
2025-10-26T18:06:24.580394Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 45 partNo : 4 messageNo: 89 size: 205040
2025-10-26T18:06:24.580435Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 45 partNo : 4 messageNo: 89 size 205040 offset: -1
2025-10-26T18:06:24.588262Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:24.590539Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.590578Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.590670Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 90  requestId: <none> cookie: 46
2025-10-26T18:06:24.593273Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.597611Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 36 partNo 0 count 3 size 8295660
2025-10-26T18:06:24.601022Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:24.605885Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.605930Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.606498Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 46 partNo : 0 messageNo: 91 size: 511959
2025-10-26T18:06:24.607264Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 46 partNo : 1 messageNo: 91 size: 511959
2025-10-26T18:06:24.607881Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 46 partNo : 2 messageNo: 91 size: 511959
2025-10-26T18:06:24.608439Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 46 partNo : 3 messageNo: 91 size: 511959
2025-10-26T18:06:24.610748Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:24.611203Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 46 partNo : 4 messageNo: 91 size: 205040
2025-10-26T18:06:24.611237Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 46 partNo : 4 messageNo: 91 size 205040 offset: -1
2025-10-26T18:06:24.612994Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.613020Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.613075Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 92  requestId: <none> cookie: 47
2025-10-26T18:06:24.614967Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 39 partNo 3 count 4 size 7988760
2025-10-26T18:06:24.617796Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.629825Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:24.632873Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.632912Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.633449Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 47 partNo : 0 messageNo: 93 size: 511959
2025-10-26T18:06:24.633992Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 47 partNo : 1 messageNo: 93 size: 511959
2025-10-26T18:06:24.634501Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 47 partNo : 2 messageNo: 93 size: 511959
2025-10-26T18:06:24.635317Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:24.635237Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 47 partNo : 3 messageNo: 93 size: 511959
2025-10-26T18:06:24.635477Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 47 partNo : 4 messageNo: 93 size: 205040
2025-10-26T18:06:24.635497Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 47 partNo : 4 messageNo: 93 size 205040 offset: -1
2025-10-26T18:06:24.636820Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.636842Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.636933Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 94  requestId: <none> cookie: 48
2025-10-26T18:06:24.638146Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.647893Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.647941Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.648487Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 48 partNo : 0 messageNo: 95 size: 511959
2025-10-26T18:06:24.649002Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 48 partNo : 1 messageNo: 95 size: 511959
2025-10-26T18:06:24.649738Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 48 partNo : 2 messageNo: 95 size: 511959
2025-10-26T18:06:24.650283Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 48 partNo : 3 messageNo: 95 size: 511959
2025-10-26T18:06:24.650538Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 48 partNo : 4 messageNo: 95 size: 205040
2025-10-26T18:06:24.650558Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 48 partNo : 4 messageNo: 95 size 205040 offset: -1
2025-10-26T18:06:24.691751Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 36 count 3 size 8295660 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:24.691795Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 39 count 4 size 7988760 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:24.691919Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 13517466 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:24.691950Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.691989Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 39, partNo: 0, Offset: 38 is stored on disk
2025-10-26T18:06:24.692001Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692021Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 39, partNo: 1, Offset: 38 is stored on disk
2025-10-26T18:06:24.692030Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692035Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 28 size 7988760
2025-10-26T18:06:24.692051Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 39, partNo: 2, Offset: 38 is stored on disk
2025-10-26T18:06:24.692058Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692074Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 36 size 8295660
2025-10-26T18:06:24.692077Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 39, partNo: 3, Offset: 38 is stored on disk
2025-10-26T18:06:24.692100Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692105Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 7 size 8295579
2025-10-26T18:06:24.692118Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 39, partNo: 4, Offset: 38 is stored on disk
2025-10-26T18:06:24.692125Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692134Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 39 size 7988760
2025-10-26T18:06:24.692142Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 40, partNo: 0, Offset: 39 is stored on disk
2025-10-26T18:06:24.692151Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692171Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 40, partNo: 1, Offset: 39 is stored on disk
2025-10-26T18:06:24.692179Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692198Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 40, partNo: 2, Offset: 39 is stored on disk
2025-10-26T18:06:24.692206Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692225Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 40, partNo: 3, Offset: 39 is stored on disk
2025-10-26T18:06:24.692236Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692257Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 40, partNo: 4, Offset: 39 is stored on disk
2025-10-26T18:06:24.692266Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692285Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 41, partNo: 0, Offset: 40 is stored on disk
2025-10-26T18:06:24.692292Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692310Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 41, partNo: 1, Offset: 40 is stored on disk
2025-10-26T18:06:24.692318Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692335Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 41, partNo: 2, Offset: 40 is stored on disk
2025-10-26T18:06:24.692346Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692364Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 41, partNo: 3, Offset: 40 is stored on disk
2025-10-26T18:06:24.692373Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692389Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 41, partNo: 4, Offset: 40 is stored on disk
2025-10-26T18:06:24.692396Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692412Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 42, partNo: 0, Offset: 41 is stored on disk
2025-10-26T18:06:24.692419Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692435Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 42, partNo: 1, Offset: 41 is stored on disk
2025-10-26T18:06:24.692440Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692457Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 42, partNo: 2, Offset: 41 is stored on disk
2025-10-26T18:06:24.692465Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692484Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 42, partNo: 3, Offset: 41 is stored on disk
2025-10-26T18:06:24.692493Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692512Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 42, partNo: 4, Offset: 41 is stored on disk
2025-10-26T18:06:24.692520Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692539Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 43, partNo: 0, Offset: 42 is stored on disk
2025-10-26T18:06:24.692545Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692564Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 43, partNo: 1, Offset: 42 is stored on disk
2025-10-26T18:06:24.692571Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692588Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 43, partNo: 2, Offset: 42 is stored on disk
2025-10-26T18:06:24.692594Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692612Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 43, partNo: 3, Offset: 42 is stored on disk
2025-10-26T18:06:24.692621Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692638Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 43, partNo: 4, Offset: 42 is stored on disk
2025-10-26T18:06:24.692644Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692660Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 44, partNo: 0, Offset: 43 is stored on disk
2025-10-26T18:06:24.692667Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692684Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 44, partNo: 1, Offset: 43 is stored on disk
2025-10-26T18:06:24.692690Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692707Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 44, partNo: 2, Offset: 43 is stored on disk
2025-10-26T18:06:24.692714Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692732Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 44, partNo: 3, Offset: 43 is stored on disk
2025-10-26T18:06:24.692740Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.692758Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 44, partNo: 4, Offset: 43 is stored on disk
2025-10-26T18:06:24.697127Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 45 partNo 0
2025-10-26T18:06:24.697191Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 45 partNo 1
2025-10-26T18:06:24.697209Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 45 partNo 2
2025-10-26T18:06:24.697227Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 45 partNo 3
2025-10-26T18:06:24.697245Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 45 partNo 4
2025-10-26T18:06:24.710281Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 45 partNo 4 FormedBlobsCount 0 NewHead: Offset 44 PartNo 0 PackedSize 2253230 count 1 nextOffset 45 batches 5
2025-10-26T18:06:24.710421Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 46 partNo 0
2025-10-26T18:06:24.710453Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 46 partNo 1
2025-10-26T18:06:24.710466Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 46 partNo 2
2025-10-26T18:06:24.710484Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 46 partNo 3
2025-10-26T18:06:24.710496Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 46 partNo 4
2025-10-26T18:06:24.721198Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 46 partNo 4 FormedBlobsCount 0 NewHead: Offset 44 PartNo 0 PackedSize 4506428 count 2 nextOffset 46 batches 9
2025-10-26T18:06:24.721340Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 47 partNo 0
2025-10-26T18:06:24.721375Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 47 partNo 1
2025-10-26T18:06:24.721393Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 47 partNo 2
2025-10-26T18:06:24.721409Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 47 partNo 3
2025-10-26T18:06:24.721427Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 47 partNo 4
2025-10-26T18:06:24.766095Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 47 partNo 4 result is x0000000000_00000000000000000043_00001_0000000003_00015 size 8295604
2025-10-26T18:06:24.766184Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000043_00001_0000000003_00015 new key d0000000000_00000000000000000043_00001_0000000003_00015 size 8295604 WTime 1761501984766
2025-10-26T18:06:24.769334Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 47 partNo 4 FormedBlobsCount 1 NewHead: Offset 46 PartNo 4 PackedSize 205118 count 1 nextOffset 47 batches 1
2025-10-26T18:06:24.772785Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 48 partNo 0
2025-10-26T18:06:24.772846Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 48 partNo 1
2025-10-26T18:06:24.772866Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 48 partNo 2
2025-10-26T18:06:24.772885Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 48 partNo 3
2025-10-26T18:06:24.772903Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 48 partNo 4
2025-10-26T18:06:24.790912Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 48 partNo 4 FormedBlobsCount 0 NewHead: Offset 46 PartNo 4 PackedSize 2458316 count 2 nextOffset 48 batches 5
2025-10-26T18:06:24.791814Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 46,2 HeadOffset 43 endOffset 44 curOffset 48 d0000000000_00000000000000000046_00004_0000000002_00004| size 2458306 WTime 1761501984791
2025-10-26T18:06:24.806894Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 77  requestId: <none> cookie: 39
2025-10-26T18:06:24.806992Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 79  requestId: <none> cookie: 40
2025-10-26T18:06:24.807031Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 81  requestId: <none> cookie: 41
2025-10-26T18:06:24.807075Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 83  requestId: <none> cookie: 42
2025-10-26T18:06:24.807129Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 85  requestId: <none> cookie: 43
2025-10-26T18:06:24.807184Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 87  requestId: <none> cookie: 44
2025-10-26T18:06:24.807235Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 7 size 8295579 cause it's been evicted from L2. Actual L1 size: 5
2025-10-26T18:06:24.807247Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 28 size 7988760 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:24.807259Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:24.813557Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:24.807644Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.807819Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.807916Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.808002Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.808063Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.808149Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.814985Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 39 offsets: 38 already_written: false write_statistics { persist_duration_ms: 498 queued_in_partition_duration_ms: 451 }
2025-10-26T18:06:24.815031Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 39
2025-10-26T18:06:24.815061Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:24.836013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 40 offsets: 39 already_written: false write_statistics { persist_duration_ms: 471 queued_in_partition_duration_ms: 447 }
2025-10-26T18:06:24.836064Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 40
2025-10-26T18:06:24.837775Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 41 offsets: 40 already_written: false write_statistics { persist_duration_ms: 356 queued_in_partition_duration_ms: 526 }
2025-10-26T18:06:24.837831Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 41
2025-10-26T18:06:24.839224Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 42 offsets: 41 already_written: false write_statistics { persist_duration_ms: 322 queued_in_partition_duration_ms: 512 }
2025-10-26T18:06:24.839261Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 42
2025-10-26T18:06:24.840777Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 43 offsets: 42 already_written: false write_statistics { persist_duration_ms: 244 queued_in_partition_duration_ms: 528 }
2025-10-26T18:06:24.840815Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 43
2025-10-26T18:06:24.842224Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 44 offsets: 43 already_written: false write_statistics { persist_duration_ms: 217 queued_in_partition_duration_ms: 299 }
2025-10-26T18:06:24.842263Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 44
2025-10-26T18:06:24.842638Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 43 partNo 1 count 3 size 8295604
2025-10-26T18:06:24.848585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 49 to 49
2025-10-26T18:06:24.855481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:24.855524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 49
2025-10-26T18:06:24.861381Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (10-10)
2025-10-26T18:06:24.861441Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 10 ProducerId: "src_id" SeqNo: 11 CreateTime: 2025-10-26T18:06:19.049000Z WriteTime: 2025-10-26T18:06:20.192000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:50340", "_ip": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:24.861757Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (11-11)
2025-10-26T18:06:24.861808Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:24.865348Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:24.870872Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:24.872092Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.872131Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.872208Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 96  requestId: <none> cookie: 49
2025-10-26T18:06:24.874799Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.891622Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.891669Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.892197Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 49 partNo : 0 messageNo: 97 size: 511959
2025-10-26T18:06:24.892700Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 49 partNo : 1 messageNo: 97 size: 511959
2025-10-26T18:06:24.893213Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 49 partNo : 2 messageNo: 97 size: 511959
2025-10-26T18:06:24.893916Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 49 partNo : 3 messageNo: 97 size: 511959
2025-10-26T18:06:24.894174Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 49 partNo : 4 messageNo: 97 size: 205040
2025-10-26T18:06:24.894195Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 49 partNo : 4 messageNo: 97 size 205040 offset: -1
2025-10-26T18:06:24.899626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 50 to 50
2025-10-26T18:06:24.902254Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:24.902295Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 50
2025-10-26T18:06:24.911156Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:24.917174Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-26T18:06:24.921278Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 43 count 3 size 8295604 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:24.921589Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 9011644 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:24.921613Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921649Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 45, partNo: 0, Offset: 44 is stored on disk
2025-10-26T18:06:24.921658Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921676Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 45, partNo: 1, Offset: 44 is stored on disk
2025-10-26T18:06:24.921702Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921719Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 45, partNo: 2, Offset: 44 is stored on disk
2025-10-26T18:06:24.921727Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921744Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 45, partNo: 3, Offset: 44 is stored on disk
2025-10-26T18:06:24.921766Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921783Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 45, partNo: 4, Offset: 44 is stored on disk
2025-10-26T18:06:24.921790Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921806Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 46, partNo: 0, Offset: 45 is stored on disk
2025-10-26T18:06:24.921815Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921833Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 46, partNo: 1, Offset: 45 is stored on disk
2025-10-26T18:06:24.921839Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921854Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 46, partNo: 2, Offset: 45 is stored on disk
2025-10-26T18:06:24.921862Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921879Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 46, partNo: 3, Offset: 45 is stored on disk
2025-10-26T18:06:24.921888Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921903Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 46, partNo: 4, Offset: 45 is stored on disk
2025-10-26T18:06:24.921912Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921926Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 47, partNo: 0, Offset: 46 is stored on disk
2025-10-26T18:06:24.921933Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.921948Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 47, partNo: 1, Offset: 46 is stored on disk
2025-10-26T18:06:24.921955Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.922212Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 47, partNo: 2, Offset: 46 is stored on disk
2025-10-26T18:06:24.922223Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.922245Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 47, partNo: 3, Offset: 46 is stored on disk
2025-10-26T18:06:24.922258Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.922276Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 47, partNo: 4, Offset: 46 is stored on disk
2025-10-26T18:06:24.922282Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.922299Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 48, partNo: 0, Offset: 47 is stored on disk
2025-10-26T18:06:24.922306Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.922323Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 48, partNo: 1, Offset: 47 is stored on disk
2025-10-26T18:06:24.922331Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.922347Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 48, partNo: 2, Offset: 47 is stored on disk
2025-10-26T18:06:24.922354Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.922370Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 48, partNo: 3, Offset: 47 is stored on disk
2025-10-26T18:06:24.922379Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:24.922395Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 48, partNo: 4, Offset: 47 is stored on disk
2025-10-26T18:06:24.928381Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 10 size 7988760
2025-10-26T18:06:24.928437Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 43 size 8295604
2025-10-26T18:06:24.932994Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 49 partNo 0
2025-10-26T18:06:24.933164Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 49 partNo 1
2025-10-26T18:06:24.933184Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 49 partNo 2
2025-10-26T18:06:24.934062Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 49 partNo 3
2025-10-26T18:06:24.934138Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 49 partNo 4
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:24.938526Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 89  requestId: <none> cookie: 45
2025-10-26T18:06:24.938620Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 91  requestId: <none> cookie: 46
2025-10-26T18:06:24.938672Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 93  requestId: <none> cookie: 47
2025-10-26T18:06:24.938722Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 95  requestId: <none> cookie: 48
2025-10-26T18:06:24.938758Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 10 size 7988760 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:24.938772Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:24.948960Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.949062Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.949096Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.949126Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:24.949197Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:24.953909Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 45 offsets: 44 already_written: false write_statistics { persist_duration_ms: 226 queued_in_partition_duration_ms: 111 }
2025-10-26T18:06:24.953952Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 45
2025-10-26T18:06:24.979921Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 46 offsets: 45 already_written: false write_statistics { persist_duration_ms: 211 queued_in_partition_duration_ms: 97 }
2025-10-26T18:06:24.980212Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 46
2025-10-26T18:06:24.991201Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 51 to 51
2025-10-26T18:06:24.990399Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:24.990438Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:24.994144Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:24.994177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 51
2025-10-26T18:06:25.002482Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (11-11)
2025-10-26T18:06:25.002565Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 11 ProducerId: "src_id" SeqNo: 12 CreateTime: 2025-10-26T18:06:19.253000Z WriteTime: 2025-10-26T18:06:20.307000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:25.007534Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (12-12)
2025-10-26T18:06:25.007614Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:25.012950Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 47 offsets: 46 already_written: false write_statistics { persist_duration_ms: 202 queued_in_partition_duration_ms: 81 }
2025-10-26T18:06:25.013004Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 47
2025-10-26T18:06:25.014313Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 48 offsets: 47 already_written: false write_statistics { persist_duration_ms: 147 queued_in_partition_duration_ms: 120 }
2025-10-26T18:06:25.014346Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 48
2025-10-26T18:06:25.026241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 52 to 52
2025-10-26T18:06:25.028059Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:25.028095Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 52
2025-10-26T18:06:25.039029Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:25.060955Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 53 to 53
2025-10-26T18:06:25.063756Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:25.063801Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 53
2025-10-26T18:06:25.064415Z :DEBUG: [/Root] Take Data. Partition 0. Read: {2, 0} (12-12)
2025-10-26T18:06:25.064464Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 12 ProducerId: "src_id" SeqNo: 13 CreateTime: 2025-10-26T18:06:19.523000Z WriteTime: 2025-10-26T18:06:20.548000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:50340", "_ip": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:25.065757Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (13-13)
2025-10-26T18:06:25.065798Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:25.066443Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 49 partNo 4 FormedBlobsCount 0 NewHead: Offset 48 PartNo 0 PackedSize 2253230 count 1 nextOffset 49 batches 5
2025-10-26T18:06:25.067449Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 48,1 HeadOffset 46 endOffset 48 curOffset 49 d0000000000_00000000000000000048_00000_0000000001_00004| size 2253220 WTime 1761501985065
2025-10-26T18:06:25.078586Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:25.082058Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 98  requestId: <none> cookie: 50
2025-10-26T18:06:25.082607Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.090525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 54 to 54
2025-10-26T18:06:25.092673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:25.092717Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 54
2025-10-26T18:06:25.093226Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-26T18:06:25.095859Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:25.105920Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:25.116562Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.116619Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.120403Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 50 partNo : 0 messageNo: 99 size: 511959
2025-10-26T18:06:25.120950Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 50 partNo : 1 messageNo: 99 size: 511959
2025-10-26T18:06:25.124347Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 50 partNo : 2 messageNo: 99 size: 511959
2025-10-26T18:06:25.127591Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 50 partNo : 3 messageNo: 99 size: 511959
2025-10-26T18:06:25.130418Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 50 partNo : 4 messageNo: 99 size: 205040
2025-10-26T18:06:25.130448Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 50 partNo : 4 messageNo: 99 size 205040 offset: -1
2025-10-26T18:06:25.131545Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.131572Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.131701Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 100  requestId: <none> cookie: 51
2025-10-26T18:06:25.132596Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.145994Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 55 to 55
2025-10-26T18:06:25.147977Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:25.148010Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 55
2025-10-26T18:06:25.148381Z :DEBUG: [/Root] Take Data. Partition 0. Read: {3, 0} (13-13)
2025-10-26T18:06:25.148417Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
2025-10-26T18:06:25.148437Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] In ContinueReadingDataImpl, ReadSizeBudget = 9011865, ReadSizeServerDelta = -8487577
2025-10-26T18:06:25.148633Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-26T18:06:25.148656Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 13 ProducerId: "src_id" SeqNo: 14 CreateTime: 2025-10-26T18:06:19.599000Z WriteTime: 2025-10-26T18:06:20.582000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:25.149940Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 1, data# { read_request { bytes_size: 9011865 } }
2025-10-26T18:06:25.150112Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 got read request: guid# 6d355c2c-3f39e7d9-3f71bbfe-c9740ef
2025-10-26T18:06:25.150173Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 performing read request: guid# aa2fd173-958a2ed1-3a5d6385-b17254ec, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 26, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-26T18:06:25.150246Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 26 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 14 EndOffset 36 ClientCommitOffset 0 committedOffset 0 Guid aa2fd173-958a2ed1-3a5d6385-b17254ec
2025-10-26T18:06:25.152299Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:25.154613Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.154649Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.155206Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 51 partNo : 0 messageNo: 101 size: 511959
2025-10-26T18:06:25.155726Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 51 partNo : 1 messageNo: 101 size: 511959
2025-10-26T18:06:25.156435Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 51 partNo : 2 messageNo: 101 size: 511959
2025-10-26T18:06:25.157049Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 51 partNo : 3 messageNo: 101 size: 511959
2025-10-26T18:06:25.157313Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 51 partNo : 4 messageNo: 101 size: 205040
2025-10-26T18:06:25.157335Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 51 partNo : 4 messageNo: 101 size 205040 offset: -1
2025-10-26T18:06:25.159092Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.159118Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.159533Z node 2 :PERSQUEUE DEBUG: Read proxy: bootstrap for direct read id: 0
2025-10-26T18:06:25.159592Z node 2 :PERSQUEUE DEBUG: read cookie 8 Topic 'rt3.dc1--test-topic' partition 0 user user offset 14 count 26 size 524288 endOffset 48 max time lag 0ms effective offset 14
2025-10-26T18:06:25.159806Z node 2 :PERSQUEUE DEBUG: read cookie 8 added 2 blobs, size 7988778 count 4 last offset 18
2025-10-26T18:06:25.159883Z node 2 :PERSQUEUE DEBUG: Reading cookie 8. Send blob request.
2025-10-26T18:06:25.159915Z node 2 :PERSQUEUE DEBUG: No blob in L1. Partition 0 offset 10 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:25.159925Z node 2 :PERSQUEUE DEBUG: No blob in L1. Partition 0 offset 14 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:25.159947Z node 2 :PERSQUEUE DEBUG: Reading cookie 8. Have to read 2 of 2 from KV
2025-10-26T18:06:25.160024Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 10
2025-10-26T18:06:25.160034Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 14
2025-10-26T18:06:25.162197Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.162228Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.162315Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 102  requestId: <none> cookie: 52
2025-10-26T18:06:25.161749Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:25.175834Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.191149Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252911 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:25.191182Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.191217Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 49, partNo: 0, Offset: 48 is stored on disk
2025-10-26T18:06:25.191228Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.191247Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 49, partNo: 1, Offset: 48 is stored on disk
2025-10-26T18:06:25.191255Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.191271Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 49, partNo: 2, Offset: 48 is stored on disk
2025-10-26T18:06:25.191279Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.191300Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 49, partNo: 3, Offset: 48 is stored on disk
2025-10-26T18:06:25.191321Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.191339Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 49, partNo: 4, Offset: 48 is stored on disk
2025-10-26T18:06:25.196853Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:25.199204Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 56 to 56
2025-10-26T18:06:25.199195Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 97  requestId: <none> cookie: 49
2025-10-26T18:06:25.203716Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:25.203761Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 56
2025-10-26T18:06:25.204378Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:25.210574Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 50 partNo 0
2025-10-26T18:06:25.210625Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 50 partNo 1
2025-10-26T18:06:25.210642Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 50 partNo 2
2025-10-26T18:06:25.210658Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 50 partNo 3
2025-10-26T18:06:25.210671Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 50 partNo 4
2025-10-26T18:06:25.244453Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.244518Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.245524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 49 offsets: 48 already_written: false write_statistics { persist_duration_ms: 257 queued_in_partition_duration_ms: 37 }
2025-10-26T18:06:25.245569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 49
2025-10-26T18:06:25.243530Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.243654Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:25.246109Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 52 partNo : 0 messageNo: 103 size: 511959
2025-10-26T18:06:25.249014Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 52 partNo : 1 messageNo: 103 size: 511959
2025-10-26T18:06:25.249514Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 52 partNo : 2 messageNo: 103 size: 511959
2025-10-26T18:06:25.250030Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 52 partNo : 3 messageNo: 103 size: 511959
2025-10-26T18:06:25.250247Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 52 partNo : 4 messageNo: 103 size: 205040
2025-10-26T18:06:25.250267Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 52 partNo : 4 messageNo: 103 size 205040 offset: -1
2025-10-26T18:06:25.256475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 57 to 57
2025-10-26T18:06:25.262233Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.263357Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:25.263394Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 57
2025-10-26T18:06:25.263893Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:25.262276Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.306947Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 58 to 58
2025-10-26T18:06:25.308683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:25.308713Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 58
2025-10-26T18:06:25.309093Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:25.317774Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 50 partNo 4 FormedBlobsCount 0 NewHead: Offset 49 PartNo 0 PackedSize 2253230 count 1 nextOffset 50 batches 5
2025-10-26T18:06:25.317914Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 51 partNo 0
2025-10-26T18:06:25.317954Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 51 partNo 1
2025-10-26T18:06:25.317978Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 51 partNo 2
2025-10-26T18:06:25.355226Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 59 to 59
2025-10-26T18:06:25.358156Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:25.358219Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 59
2025-10-26T18:06:25.358250Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:25.358750Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:25.425841Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 51 partNo 2 result is x0000000000_00000000000000000046_00004_0000000004_00014 size 7988782
2025-10-26T18:06:25.429001Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 51 partNo 3
2025-10-26T18:06:25.429069Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 51 partNo 4
2025-10-26T18:06:25.429141Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000046_00004_0000000004_00014 new key d0000000000_00000000000000000046_00004_0000000004_00014 size 7988782 WTime 1761501985427
2025-10-26T18:06:25.451544Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 51 partNo 4 FormedBlobsCount 1 NewHead: Offset 50 PartNo 2 PackedSize 1229172 count 1 nextOffset 51 batches 3
2025-10-26T18:06:25.452575Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 50,1 HeadOffset 46 endOffset 49 curOffset 51 d0000000000_00000000000000000050_00002_0000000001_00002| size 1229162 WTime 1761501985452
2025-10-26T18:06:25.455897Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 104  requestId: <none> cookie: 53
2025-10-26T18:06:25.457147Z node 2 :PERSQUEUE DEBUG: Got results. 2 of 2 from KV. Status 1
2025-10-26T18:06:25.457175Z node 2 :PERSQUEUE DEBUG: Got results. result 0 from KV. Status 0
2025-10-26T18:06:25.457188Z node 2 :PERSQUEUE DEBUG: Got results. result 1 from KV. Status 0
2025-10-26T18:06:25.457223Z node 2 :PERSQUEUE DEBUG: Prefetched blob in L1. Partition 0 offset 10 count 4 size 7988760 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:25.457240Z node 2 :PERSQUEUE DEBUG: Prefetched blob in L1. Partition 0 offset 14 count 4 size 7988778 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:25.457434Z node 2 :PERSQUEUE DEBUG: FormAnswer 2
2025-10-26T18:06:25.457729Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 32 size 7988778
2025-10-26T18:06:25.457744Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 size 7988760
2025-10-26T18:06:25.457762Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 36 size 8295660
2025-10-26T18:06:25.457773Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 size 7988778
2025-10-26T18:06:25.456716Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.481740Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:25.505574Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:25.519320Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.519367Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.519387Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 14 totakecount 4 count 0 size 1024014 from pos 0 cbcount 2
2025-10-26T18:06:25.519940Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 53 partNo : 0 messageNo: 105 size: 511959
2025-10-26T18:06:25.520545Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 53 partNo : 1 messageNo: 105 size: 511959
2025-10-26T18:06:25.521294Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 53 partNo : 2 messageNo: 105 size: 511959
2025-10-26T18:06:25.521986Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 53 partNo : 3 messageNo: 105 size: 511959
2025-10-26T18:06:25.522235Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 53 partNo : 4 messageNo: 105 size: 205040
2025-10-26T18:06:25.522264Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 53 partNo : 4 messageNo: 105 size 205040 offset: -1
2025-10-26T18:06:25.522874Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 14 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:25.523945Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.523977Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.524024Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:25.524793Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 14 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:25.527715Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 14 totakecount 4 count 1 size 717095 from pos 0 cbcount 2
2025-10-26T18:06:25.529720Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 15 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:25.532158Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 15 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:25.536933Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 15 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:25.541965Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 15 totakecount 4 count 1 size 717095 from pos 0 cbcount 2
2025-10-26T18:06:25.544848Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 46 partNo 4 count 4 size 7988782
2025-10-26T18:06:25.545251Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 16 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:25.547552Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 16 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:25.550789Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 36 size 8295660 cause it's been evicted from L2. Actual L1 size: 5
2025-10-26T18:06:25.550814Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 32 size 7988778 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:25.550830Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:25.560922Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 16 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:25.561965Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 16 totakecount 4 count 1 size 205086 from pos 0 cbcount 1
2025-10-26T18:06:25.567026Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 17 totakecount 4 count 0 size 512009 from pos 0 cbcount 1
2025-10-26T18:06:25.568636Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 17 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:25.573148Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 17 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:25.576979Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 17 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:25.580036Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 17 totakecount 4 count 1 size 205086 from pos 0 cbcount 1
2025-10-26T18:06:25.589126Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 14
2025-10-26T18:06:25.589538Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 106  requestId: <none> cookie: 54
2025-10-26T18:06:25.589893Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.621089Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:25.624477Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 46 count 4 size 7988782 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:25.624728Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505822 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:25.624735Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 39 size 7988760
2025-10-26T18:06:25.624763Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.624804Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 50, partNo: 0, Offset: 49 is stored on disk
2025-10-26T18:06:25.624817Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.624839Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 50, partNo: 1, Offset: 49 is stored on disk
2025-10-26T18:06:25.624849Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.624872Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 50, partNo: 2, Offset: 49 is stored on disk
2025-10-26T18:06:25.624881Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.624901Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 50, partNo: 3, Offset: 49 is stored on disk
2025-10-26T18:06:25.624924Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.624947Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 50, partNo: 4, Offset: 49 is stored on disk
2025-10-26T18:06:25.624956Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.624980Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 51, partNo: 0, Offset: 50 is stored on disk
2025-10-26T18:06:25.624988Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.625008Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 51, partNo: 1, Offset: 50 is stored on disk
2025-10-26T18:06:25.625016Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.625037Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 51, partNo: 2, Offset: 50 is stored on disk
2025-10-26T18:06:25.625046Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.625067Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 51, partNo: 3, Offset: 50 is stored on disk
2025-10-26T18:06:25.625082Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.625104Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 51, partNo: 4, Offset: 50 is stored on disk
2025-10-26T18:06:25.625512Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 46 size 7988782
2025-10-26T18:06:25.632863Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:25.637759Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 52 partNo 0
2025-10-26T18:06:25.637824Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 52 partNo 1
2025-10-26T18:06:25.637844Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 52 partNo 2
2025-10-26T18:06:25.637862Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 52 partNo 3
2025-10-26T18:06:25.637880Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 52 partNo 4
2025-10-26T18:06:25.667251Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 39 size 7988760 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:25.667297Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:25.673087Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.673138Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.678054Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 54 partNo : 0 messageNo: 107 size: 511959
2025-10-26T18:06:25.678557Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 52 partNo 4 FormedBlobsCount 0 NewHead: Offset 51 PartNo 0 PackedSize 2253230 count 1 nextOffset 52 batches 5
2025-10-26T18:06:25.678692Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 53 partNo 0
2025-10-26T18:06:25.678733Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 53 partNo 1
2025-10-26T18:06:25.678756Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 53 partNo 2
2025-10-26T18:06:25.678768Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 53 partNo 3
2025-10-26T18:06:25.678781Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 53 partNo 4
2025-10-26T18:06:25.679287Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 54 partNo : 1 messageNo: 107 size: 511959
2025-10-26T18:06:25.683110Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 54 partNo : 2 messageNo: 107 size: 511959
2025-10-26T18:06:25.683654Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 54 partNo : 3 messageNo: 107 size: 511959
2025-10-26T18:06:25.683900Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 54 partNo : 4 messageNo: 107 size: 205040
2025-10-26T18:06:25.683928Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 54 partNo : 4 messageNo: 107 size 205040 offset: -1
2025-10-26T18:06:25.685770Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 99  requestId: <none> cookie: 50
2025-10-26T18:06:25.686028Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 101  requestId: <none> cookie: 51
2025-10-26T18:06:25.686112Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.686141Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.693917Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 50 offsets: 49 already_written: false write_statistics { persist_duration_ms: 413 queued_in_partition_duration_ms: 80 }
2025-10-26T18:06:25.693970Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 50
2025-10-26T18:06:25.694013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:25.686393Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.686500Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.701216Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 53 partNo 4 FormedBlobsCount 0 NewHead: Offset 51 PartNo 0 PackedSize 4506428 count 2 nextOffset 53 batches 9
2025-10-26T18:06:25.702269Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 51,2 HeadOffset 50 endOffset 51 curOffset 53 d0000000000_00000000000000000051_00000_0000000002_00008| size 4506418 WTime 1761501985702
2025-10-26T18:06:25.705672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 60 to 60
2025-10-26T18:06:25.707799Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:25.707840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 60
2025-10-26T18:06:25.707868Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:25.708354Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:25.713898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 51 offsets: 50 already_written: false write_statistics { persist_duration_ms: 306 queued_in_partition_duration_ms: 160 }
2025-10-26T18:06:25.713982Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 51
2025-10-26T18:06:25.714013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:25.772316Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 61 to 61
2025-10-26T18:06:25.774537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:25.774580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 61
2025-10-26T18:06:25.774608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:25.775264Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:25.789812Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:25.789924Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 108  requestId: <none> cookie: 55
2025-10-26T18:06:25.797799Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.838327Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.838373Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.843165Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 55 partNo : 0 messageNo: 109 size: 511959
2025-10-26T18:06:25.844212Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 55 partNo : 1 messageNo: 109 size: 511959
2025-10-26T18:06:25.854444Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 49 Result { Offset: 14 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 15 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981008 CreateTimestampMS: 1761501979851 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 15 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 16 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981093 CreateTimestampMS: 1761501979903 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 16 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 17 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981108 CreateTimestampMS: 1761501979927 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 17 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 18 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981436 CreateTimestampMS: 1761501980249 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 2 BlobsFromCache: 0 SizeLag: 76507366 RealReadOffset: 17 WaitQuotaTimeMs: 0 } Cookie: 14 }
2025-10-26T18:06:25.858291Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 55 partNo : 2 messageNo: 109 size: 511959
2025-10-26T18:06:25.861924Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 55 partNo : 3 messageNo: 109 size: 511959
2025-10-26T18:06:25.862577Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 55 partNo : 4 messageNo: 109 size: 205040
2025-10-26T18:06:25.862635Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 55 partNo : 4 messageNo: 109 size 205040 offset: -1
2025-10-26T18:06:25.863936Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:25.871857Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:25.889653Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 18 endOffset 49
2025-10-26T18:06:25.889734Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 49 ReadOffset 18 ReadGuid aa2fd173-958a2ed1-3a5d6385-b17254ec has messages 1
2025-10-26T18:06:25.893410Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 18, endOffset# 49, WTime# 1761501981436, sizeLag# 76507366
2025-10-26T18:06:25.893442Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1TEvPartitionReady. Aval parts: 0
2025-10-26T18:06:25.893528Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 read done: guid# aa2fd173-958a2ed1-3a5d6385-b17254ec, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 9011864
2025-10-26T18:06:25.893563Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 response to read: guid# aa2fd173-958a2ed1-3a5d6385-b17254ec
2025-10-26T18:06:25.917320Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.917357Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.917509Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 110  requestId: <none> cookie: 56
2025-10-26T18:06:25.921089Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Process answer. Aval parts: 1
2025-10-26T18:06:25.921781Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.937561Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:25.948305Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.948350Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.948915Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 56 partNo : 0 messageNo: 111 size: 511959
2025-10-26T18:06:25.949495Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 56 partNo : 1 messageNo: 111 size: 511959
2025-10-26T18:06:25.950039Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 56 partNo : 2 messageNo: 111 size: 511959
2025-10-26T18:06:25.950550Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 56 partNo : 3 messageNo: 111 size: 511959
2025-10-26T18:06:25.950797Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 56 partNo : 4 messageNo: 111 size: 205040
2025-10-26T18:06:25.950819Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 56 partNo : 4 messageNo: 111 size 205040 offset: -1
2025-10-26T18:06:25.965461Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:25.966232Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.966267Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.966612Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 112  requestId: <none> cookie: 57
2025-10-26T18:06:25.967494Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.972373Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Got ReadResponse, serverBytesSize = 9011864, now ReadSizeBudget = 0, ReadSizeServerDelta = -8487576
2025-10-26T18:06:25.972787Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (14-14)
2025-10-26T18:06:25.972843Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 9011864 to budget
2025-10-26T18:06:25.973158Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505822 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:25.973207Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.973263Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 52, partNo: 0, Offset: 51 is stored on disk
2025-10-26T18:06:25.973281Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.973305Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 52, partNo: 1, Offset: 51 is stored on disk
2025-10-26T18:06:25.973320Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.973344Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 52, partNo: 2, Offset: 51 is stored on disk
2025-10-26T18:06:25.973357Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.973388Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 52, partNo: 3, Offset: 51 is stored on disk
2025-10-26T18:06:25.973413Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.973441Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 52, partNo: 4, Offset: 51 is stored on disk
2025-10-26T18:06:25.973455Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.973492Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 53, partNo: 0, Offset: 52 is stored on disk
2025-10-26T18:06:25.973504Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.973528Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 53, partNo: 1, Offset: 52 is stored on disk
2025-10-26T18:06:25.973539Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.973564Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 53, partNo: 2, Offset: 52 is stored on disk
2025-10-26T18:06:25.973574Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.973596Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 53, partNo: 3, Offset: 52 is stored on disk
2025-10-26T18:06:25.973610Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:25.973639Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 53, partNo: 4, Offset: 52 is stored on disk
2025-10-26T18:06:25.974914Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 103  requestId: <none> cookie: 52
2025-10-26T18:06:25.974999Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 105  requestId: <none> cookie: 53
2025-10-26T18:06:25.977128Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.977207Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:25.983557Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:25.983600Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:25.984064Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 57 partNo : 0 messageNo: 113 size: 511959
2025-10-26T18:06:25.984470Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 57 partNo : 1 messageNo: 113 size: 511959
2025-10-26T18:06:25.984861Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 57 partNo : 2 messageNo: 113 size: 511959
2025-10-26T18:06:25.985278Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 57 partNo : 3 messageNo: 113 size: 511959
2025-10-26T18:06:25.985478Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 57 partNo : 4 messageNo: 113 size: 205040
2025-10-26T18:06:25.985506Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 57 partNo : 4 messageNo: 113 size 205040 offset: -1
2025-10-26T18:06:25.992823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 52 offsets: 51 already_written: false write_statistics { persist_duration_ms: 335 queued_in_partition_duration_ms: 182 }
2025-10-26T18:06:25.992911Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 52
2025-10-26T18:06:25.992969Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:25.995416Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:25.996024Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 53 offsets: 52 already_written: false write_statistics { persist_duration_ms: 294 queued_in_partition_duration_ms: 98 }
2025-10-26T18:06:25.996056Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 53
2025-10-26T18:06:25.998845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 62 to 62
2025-10-26T18:06:26.001658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:26.001715Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 62
2025-10-26T18:06:26.002855Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (14-14)
2025-10-26T18:06:26.002906Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 14 ProducerId: "src_id" SeqNo: 15 CreateTime: 2025-10-26T18:06:19.851000Z WriteTime: 2025-10-26T18:06:21.008000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:50340", "_ip": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:26.003960Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 54 partNo 0
2025-10-26T18:06:26.004007Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 54 partNo 1
2025-10-26T18:06:26.004023Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 54 partNo 2
2025-10-26T18:06:26.004040Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 54 partNo 3
2025-10-26T18:06:26.004056Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 54 partNo 4
2025-10-26T18:06:26.004256Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (15-15)
2025-10-26T18:06:26.004434Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:26.007687Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.008176Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.008214Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.016742Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 63 to 63
2025-10-26T18:06:26.019000Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:26.019037Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 63
2025-10-26T18:06:26.019062Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:26.020071Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-26T18:06:26.020451Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 54 partNo 4 FormedBlobsCount 0 NewHead: Offset 53 PartNo 0 PackedSize 2253230 count 1 nextOffset 54 batches 5
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:26.033006Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 55 partNo 0
2025-10-26T18:06:26.055854Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 55 partNo 0 result is x0000000000_00000000000000000050_00002_0000000004_00014 size 7988800
2025-10-26T18:06:26.058344Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 55 partNo 1
2025-10-26T18:06:26.058392Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 55 partNo 2
2025-10-26T18:06:26.058408Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 55 partNo 3
2025-10-26T18:06:26.058427Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 55 partNo 4
2025-10-26T18:06:26.058489Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000050_00002_0000000004_00014 new key d0000000000_00000000000000000050_00002_0000000004_00014 size 7988800 WTime 1761501986058
2025-10-26T18:06:26.076460Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 55 partNo 4 FormedBlobsCount 1 NewHead: Offset 54 PartNo 0 PackedSize 2253230 count 1 nextOffset 55 batches 5
2025-10-26T18:06:26.076588Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 56 partNo 0
2025-10-26T18:06:26.076630Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 56 partNo 1
2025-10-26T18:06:26.076648Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 56 partNo 2
2025-10-26T18:06:26.076666Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 56 partNo 3
2025-10-26T18:06:26.076684Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 56 partNo 4
2025-10-26T18:06:26.091529Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 56 partNo 4 FormedBlobsCount 0 NewHead: Offset 54 PartNo 0 PackedSize 4506428 count 2 nextOffset 56 batches 9
2025-10-26T18:06:26.092285Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 54,2 HeadOffset 50 endOffset 53 curOffset 56 d0000000000_00000000000000000054_00000_0000000002_00008| size 4506418 WTime 1761501986092
2025-10-26T18:06:26.101483Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:26.122651Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 50 partNo 2 count 4 size 7988800
2025-10-26T18:06:26.133785Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 114  requestId: <none> cookie: 58
2025-10-26T18:06:26.135533Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.148398Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.158475Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.158534Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.159270Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 58 partNo : 0 messageNo: 115 size: 511959
2025-10-26T18:06:26.160736Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 58 partNo : 1 messageNo: 115 size: 511959
2025-10-26T18:06:26.161618Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 58 partNo : 2 messageNo: 115 size: 511959
2025-10-26T18:06:26.162324Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 58 partNo : 3 messageNo: 115 size: 511959
2025-10-26T18:06:26.162709Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 58 partNo : 4 messageNo: 115 size: 205040
2025-10-26T18:06:26.162758Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 58 partNo : 4 messageNo: 115 size 205040 offset: -1
2025-10-26T18:06:26.163301Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.164897Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.164936Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.165082Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 116  requestId: <none> cookie: 59
2025-10-26T18:06:26.165497Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.177454Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.183593Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.183640Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.184267Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 59 partNo : 0 messageNo: 117 size: 511959
2025-10-26T18:06:26.184884Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 59 partNo : 1 messageNo: 117 size: 511959
2025-10-26T18:06:26.185515Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 59 partNo : 2 messageNo: 117 size: 511959
2025-10-26T18:06:26.186213Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 59 partNo : 3 messageNo: 117 size: 511959
2025-10-26T18:06:26.186473Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 59 partNo : 4 messageNo: 117 size: 205040
2025-10-26T18:06:26.186501Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 59 partNo : 4 messageNo: 117 size 205040 offset: -1
2025-10-26T18:06:26.189447Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.203390Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.203440Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.204091Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 118  requestId: <none> cookie: 60
2025-10-26T18:06:26.204375Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.211967Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.223139Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.223194Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.232746Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 60 partNo : 0 messageNo: 119 size: 511959
2025-10-26T18:06:26.233761Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 60 partNo : 1 messageNo: 119 size: 511959
2025-10-26T18:06:26.235953Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 60 partNo : 2 messageNo: 119 size: 511959
2025-10-26T18:06:26.237518Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 60 partNo : 3 messageNo: 119 size: 511959
2025-10-26T18:06:26.238804Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.238848Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 60 partNo : 4 messageNo: 119 size: 205040
2025-10-26T18:06:26.238875Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 60 partNo : 4 messageNo: 119 size 205040 offset: -1
2025-10-26T18:06:26.247873Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.247918Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.248038Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 120  requestId: <none> cookie: 61
2025-10-26T18:06:26.248171Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 50 count 4 size 7988800 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:26.248260Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 6758733 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:26.248280Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248313Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 54, partNo: 0, Offset: 53 is stored on disk
2025-10-26T18:06:26.248323Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248343Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 54, partNo: 1, Offset: 53 is stored on disk
2025-10-26T18:06:26.248351Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248354Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 43 size 8295604
2025-10-26T18:06:26.248371Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 54, partNo: 2, Offset: 53 is stored on disk
2025-10-26T18:06:26.248379Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248382Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 50 size 7988800
2025-10-26T18:06:26.248400Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 54, partNo: 3, Offset: 53 is stored on disk
2025-10-26T18:06:26.248419Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248437Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 54, partNo: 4, Offset: 53 is stored on disk
2025-10-26T18:06:26.248445Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248463Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 55, partNo: 0, Offset: 54 is stored on disk
2025-10-26T18:06:26.248470Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248489Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 55, partNo: 1, Offset: 54 is stored on disk
2025-10-26T18:06:26.248498Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248531Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 55, partNo: 2, Offset: 54 is stored on disk
2025-10-26T18:06:26.248541Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248560Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 55, partNo: 3, Offset: 54 is stored on disk
2025-10-26T18:06:26.248573Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248591Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 55, partNo: 4, Offset: 54 is stored on disk
2025-10-26T18:06:26.248599Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248617Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 56, partNo: 0, Offset: 55 is stored on disk
2025-10-26T18:06:26.248624Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248643Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 56, partNo: 1, Offset: 55 is stored on disk
2025-10-26T18:06:26.248651Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248669Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 56, partNo: 2, Offset: 55 is stored on disk
2025-10-26T18:06:26.248678Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248697Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 56, partNo: 3, Offset: 55 is stored on disk
2025-10-26T18:06:26.248707Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.248726Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 56, partNo: 4, Offset: 55 is stored on disk
2025-10-26T18:06:26.249884Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.261364Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.266523Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.267902Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 57 partNo 0
2025-10-26T18:06:26.267957Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 57 partNo 1
2025-10-26T18:06:26.267975Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 57 partNo 2
2025-10-26T18:06:26.267991Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 57 partNo 3
2025-10-26T18:06:26.268010Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 57 partNo 4
2025-10-26T18:06:26.279817Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 57 partNo 4 FormedBlobsCount 0 NewHead: Offset 56 PartNo 0 PackedSize 2253230 count 1 nextOffset 57 batches 5
2025-10-26T18:06:26.279928Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 58 partNo 0
2025-10-26T18:06:26.279955Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 58 partNo 1
2025-10-26T18:06:26.279970Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 58 partNo 2
2025-10-26T18:06:26.279988Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 58 partNo 3
2025-10-26T18:06:26.340316Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 58 partNo 3 result is x0000000000_00000000000000000054_00000_0000000003_00015 size 8295660
2025-10-26T18:06:26.342688Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 58 partNo 4
2025-10-26T18:06:26.342755Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000054_00000_0000000003_00015 new key d0000000000_00000000000000000054_00000_0000000003_00015 size 8295660 WTime 1761501986342
2025-10-26T18:06:26.344754Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 107  requestId: <none> cookie: 54
2025-10-26T18:06:26.344850Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 109  requestId: <none> cookie: 55
2025-10-26T18:06:26.344911Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 111  requestId: <none> cookie: 56
2025-10-26T18:06:26.347308Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.347412Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.347448Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.349187Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 54 offsets: 53 already_written: false write_statistics { persist_duration_ms: 246 queued_in_partition_duration_ms: 211 }
2025-10-26T18:06:26.349230Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 54
2025-10-26T18:06:26.349256Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:26.352647Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.352711Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.353244Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 61 partNo : 0 messageNo: 121 size: 511959
2025-10-26T18:06:26.353763Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 61 partNo : 1 messageNo: 121 size: 511959
2025-10-26T18:06:26.354273Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 61 partNo : 2 messageNo: 121 size: 511959
2025-10-26T18:06:26.354740Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 61 partNo : 3 messageNo: 121 size: 511959
2025-10-26T18:06:26.354965Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 61 partNo : 4 messageNo: 121 size: 205040
2025-10-26T18:06:26.354986Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 61 partNo : 4 messageNo: 121 size 205040 offset: -1
2025-10-26T18:06:26.356426Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 43 size 8295604 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:26.356447Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:26.366351Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.366389Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.378669Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 55 offsets: 54 already_written: false write_statistics { persist_duration_ms: 189 queued_in_partition_duration_ms: 195 }
2025-10-26T18:06:26.378712Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 55
2025-10-26T18:06:26.383043Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 58 partNo 4 FormedBlobsCount 1 NewHead: Offset 57 PartNo 3 PackedSize 717145 count 1 nextOffset 58 batches 2
2025-10-26T18:06:26.383145Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 59 partNo 0
2025-10-26T18:06:26.383177Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 59 partNo 1
2025-10-26T18:06:26.383196Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 59 partNo 2
2025-10-26T18:06:26.383212Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 59 partNo 3
2025-10-26T18:06:26.383229Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 59 partNo 4
2025-10-26T18:06:26.407795Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 56 offsets: 55 already_written: false write_statistics { persist_duration_ms: 176 queued_in_partition_duration_ms: 118 }
2025-10-26T18:06:26.407829Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 56
2025-10-26T18:06:26.408451Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 59 partNo 4 FormedBlobsCount 0 NewHead: Offset 57 PartNo 3 PackedSize 2970343 count 2 nextOffset 59 batches 6
2025-10-26T18:06:26.408639Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 60 partNo 0
2025-10-26T18:06:26.408735Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 60 partNo 1
2025-10-26T18:06:26.408785Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 60 partNo 2
2025-10-26T18:06:26.408828Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 60 partNo 3
2025-10-26T18:06:26.408890Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 60 partNo 4
2025-10-26T18:06:26.429610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 64 to 64
2025-10-26T18:06:26.442657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:26.442703Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 64
2025-10-26T18:06:26.448701Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (15-15)
2025-10-26T18:06:26.448754Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 15 ProducerId: "src_id" SeqNo: 16 CreateTime: 2025-10-26T18:06:19.903000Z WriteTime: 2025-10-26T18:06:21.093000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:26.449785Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (16-16)
2025-10-26T18:06:26.449961Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:26.470004Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 65 to 65
2025-10-26T18:06:26.472156Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:26.472192Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 65
2025-10-26T18:06:26.472612Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:26.483522Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 60 partNo 4 FormedBlobsCount 0 NewHead: Offset 57 PartNo 3 PackedSize 5223541 count 3 nextOffset 60 batches 10
2025-10-26T18:06:26.484581Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 57,3 HeadOffset 54 endOffset 56 curOffset 60 d0000000000_00000000000000000057_00003_0000000003_00009| size 5223531 WTime 1761501986484
2025-10-26T18:06:26.486412Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 66 to 66
2025-10-26T18:06:26.490898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:26.490941Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 66
2025-10-26T18:06:26.490966Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:26.500100Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 122  requestId: <none> cookie: 62
2025-10-26T18:06:26.500098Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:26.502564Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.511204Z :DEBUG: [/Root] Take Data. Partition 0. Read: {2, 0} (16-16)
2025-10-26T18:06:26.511247Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 16 ProducerId: "src_id" SeqNo: 17 CreateTime: 2025-10-26T18:06:19.927000Z WriteTime: 2025-10-26T18:06:21.108000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:26.511355Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (17-17)
2025-10-26T18:06:26.511410Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:26.515651Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.527619Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.528662Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.528701Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.530258Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 62 partNo : 0 messageNo: 123 size: 511959
2025-10-26T18:06:26.530868Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 62 partNo : 1 messageNo: 123 size: 511959
2025-10-26T18:06:26.531470Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 62 partNo : 2 messageNo: 123 size: 511959
2025-10-26T18:06:26.532047Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 62 partNo : 3 messageNo: 123 size: 511959
2025-10-26T18:06:26.532363Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 62 partNo : 4 messageNo: 123 size: 205040
2025-10-26T18:06:26.532389Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 62 partNo : 4 messageNo: 123 size 205040 offset: -1
2025-10-26T18:06:26.534113Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.534137Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.534201Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 124  requestId: <none> cookie: 63
2025-10-26T18:06:26.535511Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.537664Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 54 partNo 0 count 3 size 8295660
2025-10-26T18:06:26.544596Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.544637Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.545132Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 63 partNo : 0 messageNo: 125 size: 511959
2025-10-26T18:06:26.545574Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 63 partNo : 1 messageNo: 125 size: 511959
2025-10-26T18:06:26.548964Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.551492Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 63 partNo : 2 messageNo: 125 size: 511959
2025-10-26T18:06:26.552138Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 63 partNo : 3 messageNo: 125 size: 511959
2025-10-26T18:06:26.552540Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 63 partNo : 4 messageNo: 125 size: 205040
2025-10-26T18:06:26.552564Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 63 partNo : 4 messageNo: 125 size 205040 offset: -1
2025-10-26T18:06:26.560966Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.564433Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.564491Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.565020Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 126  requestId: <none> cookie: 64
2025-10-26T18:06:26.565730Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.577921Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.577964Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.582190Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.582015Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 64 partNo : 0 messageNo: 127 size: 511959
2025-10-26T18:06:26.583715Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 64 partNo : 1 messageNo: 127 size: 511959
2025-10-26T18:06:26.584301Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 64 partNo : 2 messageNo: 127 size: 511959
2025-10-26T18:06:26.586003Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 64 partNo : 3 messageNo: 127 size: 511959
2025-10-26T18:06:26.586328Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 64 partNo : 4 messageNo: 127 size: 205040
2025-10-26T18:06:26.586353Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 64 partNo : 4 messageNo: 127 size 205040 offset: -1
2025-10-26T18:06:26.598101Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.598711Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.598750Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.598843Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 128  requestId: <none> cookie: 65
2025-10-26T18:06:26.599090Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.611212Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.617300Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.617344Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.617927Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 65 partNo : 0 messageNo: 129 size: 511959
2025-10-26T18:06:26.618470Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 65 partNo : 1 messageNo: 129 size: 511959
2025-10-26T18:06:26.620089Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.620688Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 65 partNo : 2 messageNo: 129 size: 511959
2025-10-26T18:06:26.621201Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 65 partNo : 3 messageNo: 129 size: 511959
2025-10-26T18:06:26.621420Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 65 partNo : 4 messageNo: 129 size: 205040
2025-10-26T18:06:26.621443Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 65 partNo : 4 messageNo: 129 size 205040 offset: -1
2025-10-26T18:06:26.622925Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.622955Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.623554Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 130  requestId: <none> cookie: 66
2025-10-26T18:06:26.623815Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.626431Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 54 count 3 size 8295660 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:26.626563Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 10 size 7988760
2025-10-26T18:06:26.626596Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 54 size 8295660
2025-10-26T18:06:26.626639Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 9011644 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:26.626672Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.626745Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 57, partNo: 0, Offset: 56 is stored on disk
2025-10-26T18:06:26.626775Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.626807Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 57, partNo: 1, Offset: 56 is stored on disk
2025-10-26T18:06:26.626832Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.626894Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 57, partNo: 2, Offset: 56 is stored on disk
2025-10-26T18:06:26.626920Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.626964Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 57, partNo: 3, Offset: 56 is stored on disk
2025-10-26T18:06:26.627001Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627057Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 57, partNo: 4, Offset: 56 is stored on disk
2025-10-26T18:06:26.627079Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627111Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 58, partNo: 0, Offset: 57 is stored on disk
2025-10-26T18:06:26.627132Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627174Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 58, partNo: 1, Offset: 57 is stored on disk
2025-10-26T18:06:26.627197Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627237Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 58, partNo: 2, Offset: 57 is stored on disk
2025-10-26T18:06:26.627274Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627315Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 58, partNo: 3, Offset: 57 is stored on disk
2025-10-26T18:06:26.627333Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627381Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 58, partNo: 4, Offset: 57 is stored on disk
2025-10-26T18:06:26.627407Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627445Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 59, partNo: 0, Offset: 58 is stored on disk
2025-10-26T18:06:26.627463Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627498Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 59, partNo: 1, Offset: 58 is stored on disk
2025-10-26T18:06:26.627520Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627567Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 59, partNo: 2, Offset: 58 is stored on disk
2025-10-26T18:06:26.627588Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627631Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 59, partNo: 3, Offset: 58 is stored on disk
2025-10-26T18:06:26.627655Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627688Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 59, partNo: 4, Offset: 58 is stored on disk
2025-10-26T18:06:26.627708Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627749Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 60, partNo: 0, Offset: 59 is stored on disk
2025-10-26T18:06:26.627770Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627816Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 60, partNo: 1, Offset: 59 is stored on disk
2025-10-26T18:06:26.627831Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627866Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 60, partNo: 2, Offset: 59 is stored on disk
2025-10-26T18:06:26.627887Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627926Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 60, partNo: 3, Offset: 59 is stored on disk
2025-10-26T18:06:26.627948Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:26.627989Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 60, partNo: 4, Offset: 59 is stored on disk
2025-10-26T18:06:26.630533Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 113  requestId: <none> cookie: 57
2025-10-26T18:06:26.630629Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 115  requestId: <none> cookie: 58
2025-10-26T18:06:26.630686Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 117  requestId: <none> cookie: 59
2025-10-26T18:06:26.630742Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 119  requestId: <none> cookie: 60
2025-10-26T18:06:26.630803Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 10 size 7988760 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:26.630818Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:26.637914Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.638051Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.638120Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.638183Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.639789Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 57 offsets: 56 already_written: false write_statistics { persist_duration_ms: 359 queued_in_partition_duration_ms: 133 }
2025-10-26T18:06:26.639854Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 57
2025-10-26T18:06:26.639890Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:26.652029Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 67 to 67
2025-10-26T18:06:26.657019Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.657069Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.662224Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 66 partNo : 0 messageNo: 131 size: 511959
2025-10-26T18:06:26.668568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:26.668644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 67
2025-10-26T18:06:26.668810Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 66 partNo : 1 messageNo: 131 size: 511959
2025-10-26T18:06:26.669458Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 66 partNo : 2 messageNo: 131 size: 511959
2025-10-26T18:06:26.670911Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 61 partNo 0
2025-10-26T18:06:26.670957Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 61 partNo 1
2025-10-26T18:06:26.670973Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 61 partNo 2
2025-10-26T18:06:26.670990Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 61 partNo 3
2025-10-26T18:06:26.671006Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 61 partNo 4
2025-10-26T18:06:26.673671Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:26.674128Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:26.678135Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 66 partNo : 3 messageNo: 131 size: 511959
2025-10-26T18:06:26.678293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 58 offsets: 57 already_written: false write_statistics { persist_duration_ms: 346 queued_in_partition_duration_ms: 114 }
2025-10-26T18:06:26.678353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 58
2025-10-26T18:06:26.678379Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:26.678538Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 66 partNo : 4 messageNo: 131 size: 205040
2025-10-26T18:06:26.678588Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 66 partNo : 4 messageNo: 131 size 205040 offset: -1
2025-10-26T18:06:26.681271Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 59 offsets: 58 already_written: false write_statistics { persist_duration_ms: 243 queued_in_partition_duration_ms: 180 }
2025-10-26T18:06:26.681318Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 59
2025-10-26T18:06:26.687117Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 60 offsets: 59 already_written: false write_statistics { persist_duration_ms: 217 queued_in_partition_duration_ms: 168 }
2025-10-26T18:06:26.687173Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 60
2025-10-26T18:06:26.694718Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.700025Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.700736Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.700773Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.707109Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 68 to 68
2025-10-26T18:06:26.709074Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:26.709117Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 68
2025-10-26T18:06:26.714225Z :DEBUG: [/Root] Take Data. Partition 0. Read: {3, 0} (17-17)
2025-10-26T18:06:26.714285Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
2025-10-26T18:06:26.714315Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] In ContinueReadingDataImpl, ReadSizeBudget = 9011864, ReadSizeServerDelta = -8487576
2025-10-26T18:06:26.714550Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-26T18:06:26.714587Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 17 ProducerId: "src_id" SeqNo: 18 CreateTime: 2025-10-26T18:06:20.249000Z WriteTime: 2025-10-26T18:06:21.436000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:26.722886Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 1, data# { read_request { bytes_size: 9011864 } }
2025-10-26T18:06:26.722987Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 got read request: guid# 5bd61ee5-c660a94c-83fa8d41-5d4c11f2
2025-10-26T18:06:26.723041Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 performing read request: guid# 7db576af-3957b843-2c3fb3b6-ec1f3582, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 37, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-26T18:06:26.723119Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 37 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 18 EndOffset 49 ClientCommitOffset 0 committedOffset 0 Guid 7db576af-3957b843-2c3fb3b6-ec1f3582
2025-10-26T18:06:26.723684Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.723722Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.723791Z node 2 :PERSQUEUE DEBUG: Read proxy: bootstrap for direct read id: 0
2025-10-26T18:06:26.732089Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 69 to 69
2025-10-26T18:06:26.734860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:26.734903Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 69
2025-10-26T18:06:26.739965Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:26.742488Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 61 partNo 4 FormedBlobsCount 0 NewHead: Offset 60 PartNo 0 PackedSize 2253230 count 1 nextOffset 61 batches 5
2025-10-26T18:06:26.742620Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 62 partNo 0
2025-10-26T18:06:26.742658Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 62 partNo 1
2025-10-26T18:06:26.750128Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 70 to 70
2025-10-26T18:06:26.752090Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:26.752132Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 70
2025-10-26T18:06:26.752161Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:26.752984Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:26.783443Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 62 partNo 1 result is x0000000000_00000000000000000057_00003_0000000004_00014 size 7988782
2025-10-26T18:06:26.784497Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 62 partNo 2
2025-10-26T18:06:26.784527Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 62 partNo 3
2025-10-26T18:06:26.784540Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 62 partNo 4
2025-10-26T18:06:26.784587Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000057_00003_0000000004_00014 new key d0000000000_00000000000000000057_00003_0000000004_00014 size 7988782 WTime 1761501986783
2025-10-26T18:06:26.802300Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 62 partNo 4 FormedBlobsCount 1 NewHead: Offset 61 PartNo 1 PackedSize 1741199 count 1 nextOffset 62 batches 4
2025-10-26T18:06:26.802399Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 63 partNo 0
2025-10-26T18:06:26.802422Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 63 partNo 1
2025-10-26T18:06:26.802434Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 63 partNo 2
2025-10-26T18:06:26.802447Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 63 partNo 3
2025-10-26T18:06:26.802461Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 63 partNo 4
2025-10-26T18:06:26.823505Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 63 partNo 4 FormedBlobsCount 0 NewHead: Offset 61 PartNo 1 PackedSize 3994397 count 2 nextOffset 63 batches 8
2025-10-26T18:06:26.823640Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 64 partNo 0
2025-10-26T18:06:26.823683Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 64 partNo 1
2025-10-26T18:06:26.823702Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 64 partNo 2
2025-10-26T18:06:26.823720Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 64 partNo 3
2025-10-26T18:06:26.823736Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 64 partNo 4
2025-10-26T18:06:26.840428Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 64 partNo 4 FormedBlobsCount 0 NewHead: Offset 61 PartNo 1 PackedSize 6247595 count 3 nextOffset 64 batches 12
2025-10-26T18:06:26.840552Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 65 partNo 0
2025-10-26T18:06:26.840583Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 65 partNo 1
2025-10-26T18:06:26.840603Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 65 partNo 2
2025-10-26T18:06:26.840621Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 65 partNo 3
2025-10-26T18:06:26.840639Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 65 partNo 4
2025-10-26T18:06:26.891181Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 65 partNo 4 result is x0000000000_00000000000000000061_00001_0000000003_00015 size 8295582
2025-10-26T18:06:26.891272Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000061_00001_0000000003_00015 new key d0000000000_00000000000000000061_00001_0000000003_00015 size 8295582 WTime 1761501986891
2025-10-26T18:06:26.896082Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 65 partNo 4 FormedBlobsCount 1 NewHead: Offset 64 PartNo 4 PackedSize 205118 count 1 nextOffset 65 batches 1
2025-10-26T18:06:26.899320Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 64,1 HeadOffset 57 endOffset 60 curOffset 65 d0000000000_00000000000000000064_00004_0000000001_00000| size 205108 WTime 1761501986899
2025-10-26T18:06:26.899898Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:26.900574Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 132  requestId: <none> cookie: 67
2025-10-26T18:06:26.900731Z node 2 :PERSQUEUE DEBUG: read cookie 9 Topic 'rt3.dc1--test-topic' partition 0 user user offset 18 count 37 size 524288 endOffset 60 max time lag 0ms effective offset 18
2025-10-26T18:06:26.900819Z node 2 :PERSQUEUE DEBUG: read cookie 9 added 1 blobs, size 8295638 count 3 last offset 21
2025-10-26T18:06:26.900856Z node 2 :PERSQUEUE DEBUG: Reading cookie 9. Send blob request.
2025-10-26T18:06:26.901350Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.913298Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.913354Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.913962Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 67 partNo : 0 messageNo: 133 size: 511959
2025-10-26T18:06:26.917187Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.914515Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 67 partNo : 1 messageNo: 133 size: 511959
2025-10-26T18:06:26.915060Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 67 partNo : 2 messageNo: 133 size: 511959
2025-10-26T18:06:26.915585Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 67 partNo : 3 messageNo: 133 size: 511959
2025-10-26T18:06:26.915846Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 67 partNo : 4 messageNo: 133 size: 205040
2025-10-26T18:06:26.915870Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 67 partNo : 4 messageNo: 133 size 205040 offset: -1
2025-10-26T18:06:26.928543Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.929137Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.929182Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.929307Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 134  requestId: <none> cookie: 68
2025-10-26T18:06:26.930565Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.936442Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 57 partNo 3 count 4 size 7988782
2025-10-26T18:06:26.956597Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.956757Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.956642Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.957641Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 68 partNo : 0 messageNo: 135 size: 511959
2025-10-26T18:06:26.958113Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 68 partNo : 1 messageNo: 135 size: 511959
2025-10-26T18:06:26.958493Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 68 partNo : 2 messageNo: 135 size: 511959
2025-10-26T18:06:26.958889Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 68 partNo : 3 messageNo: 135 size: 511959
2025-10-26T18:06:26.959115Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 68 partNo : 4 messageNo: 135 size: 205040
2025-10-26T18:06:26.959144Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 68 partNo : 4 messageNo: 135 size 205040 offset: -1
2025-10-26T18:06:26.960984Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 61 partNo 1 count 3 size 8295582
2025-10-26T18:06:26.968519Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.969015Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.969079Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.969229Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 136  requestId: <none> cookie: 69
2025-10-26T18:06:26.969985Z node 2 :PERSQUEUE DEBUG: No blob in L1. Partition 0 offset 18 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:26.970028Z node 2 :PERSQUEUE DEBUG: Reading cookie 9. Have to read 1 of 1 from KV
2025-10-26T18:06:26.970070Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 18
2025-10-26T18:06:26.969472Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:26.978312Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:26.981219Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.981261Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.981809Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 69 partNo : 0 messageNo: 137 size: 511959
2025-10-26T18:06:26.983732Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 69 partNo : 1 messageNo: 137 size: 511959
2025-10-26T18:06:26.984393Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 69 partNo : 2 messageNo: 137 size: 511959
2025-10-26T18:06:26.984847Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 69 partNo : 3 messageNo: 137 size: 511959
2025-10-26T18:06:26.985091Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 69 partNo : 4 messageNo: 137 size: 205040
2025-10-26T18:06:26.985119Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 69 partNo : 4 messageNo: 137 size 205040 offset: -1
2025-10-26T18:06:26.989921Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:26.990657Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:26.990694Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:26.990783Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 138  requestId: <none> cookie: 70
2025-10-26T18:06:26.991125Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.000049Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.000122Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.002325Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 70 partNo : 0 messageNo: 139 size: 511959
2025-10-26T18:06:27.002931Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 70 partNo : 1 messageNo: 139 size: 511959
2025-10-26T18:06:27.003582Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 70 partNo : 2 messageNo: 139 size: 511959
2025-10-26T18:06:27.004120Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 70 partNo : 3 messageNo: 139 size: 511959
2025-10-26T18:06:27.004621Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 70 partNo : 4 messageNo: 139 size: 205040
2025-10-26T18:06:27.004657Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 70 partNo : 4 messageNo: 139 size 205040 offset: -1
2025-10-26T18:06:27.064927Z node 2 :PERSQUEUE DEBUG: Got results. 1 of 1 from KV. Status 1
2025-10-26T18:06:27.064959Z node 2 :PERSQUEUE DEBUG: Got results. result 0 from KV. Status 0
2025-10-26T18:06:27.064995Z node 2 :PERSQUEUE DEBUG: Prefetched blob in L1. Partition 0 offset 18 count 3 size 8295638 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:27.065192Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 14 size 7988778
2025-10-26T18:06:27.065300Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 size 8295638
2025-10-26T18:06:27.065760Z node 2 :PERSQUEUE DEBUG: FormAnswer 1
2025-10-26T18:06:27.068374Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 topic 'rt3.dc1--test-topicCounters. CacheSize 40557658 CachedBlobs 5
2025-10-26T18:06:27.068535Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 57 count 4 size 7988782 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:27.068559Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 61 count 3 size 8295582 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:27.068621Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 14 size 7988778 cause it's been evicted from L2. Actual L1 size: 6
2025-10-26T18:06:27.068639Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:27.069738Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 46 size 7988782
2025-10-26T18:06:27.069796Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 57 size 7988782
2025-10-26T18:06:27.070686Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 50 size 7988800
2025-10-26T18:06:27.070762Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 61 size 8295582
2025-10-26T18:06:27.072617Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 50 size 7988800 cause it's been evicted from L2. Actual L1 size: 5
2025-10-26T18:06:27.072648Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 46 size 7988782 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:27.072666Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:27.074163Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 18 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-26T18:06:27.079376Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 18 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:27.080921Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 18 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:27.082491Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 18 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:27.084511Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 18 totakecount 3 count 1 size 717095 from pos 0 cbcount 2
2025-10-26T18:06:27.086256Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 19 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:27.087792Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 19 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:27.089263Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 19 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:27.091212Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 19 totakecount 3 count 1 size 717095 from pos 0 cbcount 2
2025-10-26T18:06:27.092989Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 20 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:27.094725Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 20 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:27.096218Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 20 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:27.097309Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 20 totakecount 3 count 1 size 205086 from pos 0 cbcount 1
2025-10-26T18:06:27.100984Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 21 totakecount 3 count 0 size 1536000 from pos 0 cbcount 3
2025-10-26T18:06:27.103321Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 11264555 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:27.103354Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103392Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 61, partNo: 0, Offset: 60 is stored on disk
2025-10-26T18:06:27.103404Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103424Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 61, partNo: 1, Offset: 60 is stored on disk
2025-10-26T18:06:27.103433Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103454Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 61, partNo: 2, Offset: 60 is stored on disk
2025-10-26T18:06:27.103463Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103485Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 61, partNo: 3, Offset: 60 is stored on disk
2025-10-26T18:06:27.103508Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103526Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 61, partNo: 4, Offset: 60 is stored on disk
2025-10-26T18:06:27.103533Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103552Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 62, partNo: 0, Offset: 61 is stored on disk
2025-10-26T18:06:27.103560Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103585Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 62, partNo: 1, Offset: 61 is stored on disk
2025-10-26T18:06:27.103592Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103612Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 62, partNo: 2, Offset: 61 is stored on disk
2025-10-26T18:06:27.103621Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103638Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 62, partNo: 3, Offset: 61 is stored on disk
2025-10-26T18:06:27.103647Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103667Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 62, partNo: 4, Offset: 61 is stored on disk
2025-10-26T18:06:27.103677Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103699Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 63, partNo: 0, Offset: 62 is stored on disk
2025-10-26T18:06:27.103706Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103725Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 63, partNo: 1, Offset: 62 is stored on disk
2025-10-26T18:06:27.103734Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103754Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 63, partNo: 2, Offset: 62 is stored on disk
2025-10-26T18:06:27.103761Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103780Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 63, partNo: 3, Offset: 62 is stored on disk
2025-10-26T18:06:27.103790Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103810Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 63, partNo: 4, Offset: 62 is stored on disk
2025-10-26T18:06:27.103817Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103835Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 64, partNo: 0, Offset: 63 is stored on disk
2025-10-26T18:06:27.103843Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103859Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 64, partNo: 1, Offset: 63 is stored on disk
2025-10-26T18:06:27.103866Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103882Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 64, partNo: 2, Offset: 63 is stored on disk
2025-10-26T18:06:27.103889Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103907Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 64, partNo: 3, Offset: 63 is stored on disk
2025-10-26T18:06:27.103915Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103935Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 64, partNo: 4, Offset: 63 is stored on disk
2025-10-26T18:06:27.103943Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103962Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 65, partNo: 0, Offset: 64 is stored on disk
2025-10-26T18:06:27.103969Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.103986Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 65, partNo: 1, Offset: 64 is stored on disk
2025-10-26T18:06:27.103993Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.104012Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 65, partNo: 2, Offset: 64 is stored on disk
2025-10-26T18:06:27.104018Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.104034Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 65, partNo: 3, Offset: 64 is stored on disk
2025-10-26T18:06:27.104045Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.104063Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 65, partNo: 4, Offset: 64 is stored on disk
2025-10-26T18:06:27.107251Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 66 partNo 0
2025-10-26T18:06:27.107303Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 66 partNo 1
2025-10-26T18:06:27.107321Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 66 partNo 2
2025-10-26T18:06:27.107338Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 66 partNo 3
2025-10-26T18:06:27.107362Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 66 partNo 4
2025-10-26T18:06:27.115440Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18
2025-10-26T18:06:27.126457Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 66 partNo 4 FormedBlobsCount 0 NewHead: Offset 65 PartNo 0 PackedSize 2253230 count 1 nextOffset 66 batches 5
2025-10-26T18:06:27.126596Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 67 partNo 0
2025-10-26T18:06:27.126635Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 67 partNo 1
2025-10-26T18:06:27.126668Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 67 partNo 2
2025-10-26T18:06:27.126685Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 67 partNo 3
2025-10-26T18:06:27.126703Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 67 partNo 4
2025-10-26T18:06:27.155478Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 121  requestId: <none> cookie: 61
2025-10-26T18:06:27.155598Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 123  requestId: <none> cookie: 62
2025-10-26T18:06:27.155665Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 125  requestId: <none> cookie: 63
2025-10-26T18:06:27.155735Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 127  requestId: <none> cookie: 64
2025-10-26T18:06:27.155802Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 129  requestId: <none> cookie: 65
2025-10-26T18:06:27.157452Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 67 partNo 4 FormedBlobsCount 0 NewHead: Offset 65 PartNo 0 PackedSize 4506428 count 2 nextOffset 67 batches 9
2025-10-26T18:06:27.157563Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 68 partNo 0
2025-10-26T18:06:27.157589Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 68 partNo 1
2025-10-26T18:06:27.157606Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 68 partNo 2
2025-10-26T18:06:27.157625Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 68 partNo 3
2025-10-26T18:06:27.157640Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 68 partNo 4
2025-10-26T18:06:27.166308Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.166446Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.166491Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.166537Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.166581Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.167603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 61 offsets: 60 already_written: false write_statistics { persist_duration_ms: 432 queued_in_partition_duration_ms: 170 }
2025-10-26T18:06:27.167689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 61
2025-10-26T18:06:27.167751Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:27.170021Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 62 offsets: 61 already_written: false write_statistics { persist_duration_ms: 360 queued_in_partition_duration_ms: 208 }
2025-10-26T18:06:27.171902Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 62
2025-10-26T18:06:27.184304Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 63 offsets: 62 already_written: false write_statistics { persist_duration_ms: 301 queued_in_partition_duration_ms: 249 }
2025-10-26T18:06:27.184344Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 63
2025-10-26T18:06:27.185503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 64 offsets: 63 already_written: false write_statistics { persist_duration_ms: 279 queued_in_partition_duration_ms: 235 }
2025-10-26T18:06:27.185524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 64
2025-10-26T18:06:27.187264Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 65 offsets: 64 already_written: false write_statistics { persist_duration_ms: 262 queued_in_partition_duration_ms: 217 }
2025-10-26T18:06:27.187305Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 65
2025-10-26T18:06:27.199200Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 71 to 71
2025-10-26T18:06:27.201120Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.201168Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 71
2025-10-26T18:06:27.207606Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:27.217613Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 68 partNo 4 FormedBlobsCount 0 NewHead: Offset 65 PartNo 0 PackedSize 6759626 count 3 nextOffset 68 batches 13
2025-10-26T18:06:27.219375Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:27.219884Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 69 partNo 0
2025-10-26T18:06:27.219936Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 69 partNo 1
2025-10-26T18:06:27.219959Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 69 partNo 2
2025-10-26T18:06:27.231604Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:27.234060Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.234111Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.235589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 72 to 72
2025-10-26T18:06:27.238970Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.239016Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 72
2025-10-26T18:06:27.245177Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:27.266526Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 60 Result { Offset: 18 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 19 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981469 CreateTimestampMS: 1761501980286 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 19 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 20 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981529 CreateTimestampMS: 1761501980314 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 20 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 21 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981550 CreateTimestampMS: 1761501980641 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 1 BlobsFromCache: 0 SizeLag: 86338988 RealReadOffset: 20 WaitQuotaTimeMs: 0 } Cookie: 18 }
2025-10-26T18:06:27.274078Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 73 to 73
2025-10-26T18:06:27.277066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.277117Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 73
2025-10-26T18:06:27.285276Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 21 endOffset 60
2025-10-26T18:06:27.285370Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 60 ReadOffset 21 ReadGuid 7db576af-3957b843-2c3fb3b6-ec1f3582 has messages 1
2025-10-26T18:06:27.285426Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 21, endOffset# 60, WTime# 1761501981550, sizeLag# 86338988
2025-10-26T18:06:27.285445Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1TEvPartitionReady. Aval parts: 0
2025-10-26T18:06:27.285521Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 read done: guid# 7db576af-3957b843-2c3fb3b6-ec1f3582, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 6758905
2025-10-26T18:06:27.285550Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 response to read: guid# 7db576af-3957b843-2c3fb3b6-ec1f3582
2025-10-26T18:06:27.287788Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:27.314381Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Process answer. Aval parts: 1
2025-10-26T18:06:27.338713Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Got ReadResponse, serverBytesSize = 6758905, now ReadSizeBudget = 0, ReadSizeServerDelta = -6234617
2025-10-26T18:06:27.338996Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (18-18)
2025-10-26T18:06:27.339033Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 6758905 to budget
2025-10-26T18:06:27.342758Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 74 to 74
2025-10-26T18:06:27.351099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.351148Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 74
2025-10-26T18:06:27.355629Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (18-18)
2025-10-26T18:06:27.355695Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 18 ProducerId: "src_id" SeqNo: 19 CreateTime: 2025-10-26T18:06:20.286000Z WriteTime: 2025-10-26T18:06:21.469000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:27.357766Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (19-19)
2025-10-26T18:06:27.357827Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:27.367245Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 75 to 75
2025-10-26T18:06:27.370177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.370244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 75
2025-10-26T18:06:27.370282Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:27.371908Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:27.395996Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 69 partNo 2 result is x0000000000_00000000000000000064_00004_0000000004_00014 size 7988760
2025-10-26T18:06:27.397476Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 69 partNo 3
2025-10-26T18:06:27.397538Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 69 partNo 4
2025-10-26T18:06:27.397608Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000064_00004_0000000004_00014 new key d0000000000_00000000000000000064_00004_0000000004_00014 size 7988760 WTime 1761501987394
2025-10-26T18:06:27.411274Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 69 partNo 4 FormedBlobsCount 1 NewHead: Offset 68 PartNo 2 PackedSize 1229172 count 1 nextOffset 69 batches 3
2025-10-26T18:06:27.411387Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 70 partNo 0
2025-10-26T18:06:27.411423Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 70 partNo 1
2025-10-26T18:06:27.411442Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 70 partNo 2
2025-10-26T18:06:27.411462Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 70 partNo 3
2025-10-26T18:06:27.411478Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 70 partNo 4
2025-10-26T18:06:27.423457Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 70 partNo 4 FormedBlobsCount 0 NewHead: Offset 68 PartNo 2 PackedSize 3482370 count 2 nextOffset 70 batches 7
2025-10-26T18:06:27.424328Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 68,2 HeadOffset 64 endOffset 65 curOffset 70 d0000000000_00000000000000000068_00002_0000000002_00006| size 3482360 WTime 1761501987423
2025-10-26T18:06:27.436876Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:27.452950Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 64 partNo 4 count 4 size 7988760
2025-10-26T18:06:27.458806Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 140  requestId: <none> cookie: 71
2025-10-26T18:06:27.459227Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.468281Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:27.477079Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.477125Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.478164Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 71 partNo : 0 messageNo: 141 size: 511959
2025-10-26T18:06:27.480698Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:27.481920Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 71 partNo : 1 messageNo: 141 size: 511959
2025-10-26T18:06:27.482512Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 71 partNo : 2 messageNo: 141 size: 511959
2025-10-26T18:06:27.486222Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 71 partNo : 3 messageNo: 141 size: 511959
2025-10-26T18:06:27.486484Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 71 partNo : 4 messageNo: 141 size: 205040
2025-10-26T18:06:27.486506Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 71 partNo : 4 messageNo: 141 size 205040 offset: -1
2025-10-26T18:06:27.490786Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.490823Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.491371Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 142  requestId: <none> cookie: 72
2025-10-26T18:06:27.491632Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.497936Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 64 count 4 size 7988760 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:27.498094Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 11264555 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:27.498114Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498153Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 66, partNo: 0, Offset: 65 is stored on disk
2025-10-26T18:06:27.498165Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498188Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 66, partNo: 1, Offset: 65 is stored on disk
2025-10-26T18:06:27.498197Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498219Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 66, partNo: 2, Offset: 65 is stored on disk
2025-10-26T18:06:27.498227Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498248Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 66, partNo: 3, Offset: 65 is stored on disk
2025-10-26T18:06:27.498268Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498290Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 66, partNo: 4, Offset: 65 is stored on disk
2025-10-26T18:06:27.498299Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498318Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 67, partNo: 0, Offset: 66 is stored on disk
2025-10-26T18:06:27.498326Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498346Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 67, partNo: 1, Offset: 66 is stored on disk
2025-10-26T18:06:27.498355Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498373Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 67, partNo: 2, Offset: 66 is stored on disk
2025-10-26T18:06:27.498379Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498398Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 67, partNo: 3, Offset: 66 is stored on disk
2025-10-26T18:06:27.498411Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498430Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 67, partNo: 4, Offset: 66 is stored on disk
2025-10-26T18:06:27.498438Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498455Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 68, partNo: 0, Offset: 67 is stored on disk
2025-10-26T18:06:27.498463Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498481Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 68, partNo: 1, Offset: 67 is stored on disk
2025-10-26T18:06:27.498488Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498508Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 68, partNo: 2, Offset: 67 is stored on disk
2025-10-26T18:06:27.498517Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498538Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 68, partNo: 3, Offset: 67 is stored on disk
2025-10-26T18:06:27.498549Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498567Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 68, partNo: 4, Offset: 67 is stored on disk
2025-10-26T18:06:27.498574Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498591Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 69, partNo: 0, Offset: 68 is stored on disk
2025-10-26T18:06:27.498598Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498617Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 69, partNo: 1, Offset: 68 is stored on disk
2025-10-26T18:06:27.498624Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498642Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 69, partNo: 2, Offset: 68 is stored on disk
2025-10-26T18:06:27.498648Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498666Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 69, partNo: 3, Offset: 68 is stored on disk
2025-10-26T18:06:27.498677Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498694Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 69, partNo: 4, Offset: 68 is stored on disk
2025-10-26T18:06:27.498702Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498719Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 70, partNo: 0, Offset: 69 is stored on disk
2025-10-26T18:06:27.498726Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498746Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 70, partNo: 1, Offset: 69 is stored on disk
2025-10-26T18:06:27.498753Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498769Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 70, partNo: 2, Offset: 69 is stored on disk
2025-10-26T18:06:27.498776Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498793Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 70, partNo: 3, Offset: 69 is stored on disk
2025-10-26T18:06:27.498802Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.498820Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 70, partNo: 4, Offset: 69 is stored on disk
2025-10-26T18:06:27.503406Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 54 size 8295660
2025-10-26T18:06:27.503459Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 64 size 7988760
2025-10-26T18:06:27.515560Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:27.531656Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 71 partNo 0
2025-10-26T18:06:27.531713Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 71 partNo 1
2025-10-26T18:06:27.531734Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 71 partNo 2
2025-10-26T18:06:27.531752Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 71 partNo 3
2025-10-26T18:06:27.531771Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 71 partNo 4
2025-10-26T18:06:27.533073Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:27.554877Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 71 partNo 4 FormedBlobsCount 0 NewHead: Offset 70 PartNo 0 PackedSize 2253230 count 1 nextOffset 71 batches 5
2025-10-26T18:06:27.555822Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 70,1 HeadOffset 68 endOffset 70 curOffset 71 d0000000000_00000000000000000070_00000_0000000001_00004| size 2253220 WTime 1761501987555
2025-10-26T18:06:27.560800Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 131  requestId: <none> cookie: 66
2025-10-26T18:06:27.560908Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 133  requestId: <none> cookie: 67
2025-10-26T18:06:27.560966Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 135  requestId: <none> cookie: 68
2025-10-26T18:06:27.561041Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 137  requestId: <none> cookie: 69
2025-10-26T18:06:27.561120Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 139  requestId: <none> cookie: 70
2025-10-26T18:06:27.561345Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.561476Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.561535Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.561570Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.561604Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.564439Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 66 offsets: 65 already_written: false write_statistics { persist_duration_ms: 390 queued_in_partition_duration_ms: 206 }
2025-10-26T18:06:27.564501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 66
2025-10-26T18:06:27.564542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:27.564889Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.564959Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.565724Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 72 partNo : 0 messageNo: 143 size: 511959
2025-10-26T18:06:27.566539Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 72 partNo : 1 messageNo: 143 size: 511959
2025-10-26T18:06:27.566637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 67 offsets: 66 already_written: false write_statistics { persist_duration_ms: 371 queued_in_partition_duration_ms: 205 }
2025-10-26T18:06:27.566679Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 67
2025-10-26T18:06:27.567216Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 72 partNo : 2 messageNo: 143 size: 511959
2025-10-26T18:06:27.568534Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 72 partNo : 3 messageNo: 143 size: 511959
2025-10-26T18:06:27.568806Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 72 partNo : 4 messageNo: 143 size: 205040
2025-10-26T18:06:27.568827Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 72 partNo : 4 messageNo: 143 size 205040 offset: -1
2025-10-26T18:06:27.572674Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 76 to 76
2025-10-26T18:06:27.578747Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.578793Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 76
2025-10-26T18:06:27.579841Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (19-19)
2025-10-26T18:06:27.579891Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
2025-10-26T18:06:27.579953Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 68 offsets: 67 already_written: false write_statistics { persist_duration_ms: 340 queued_in_partition_duration_ms: 197 }
>>> Got event: 2025-10-26T18:06:27.580002Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 68
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 19 ProducerId: "src_id" SeqNo: 20 CreateTime: 2025-10-26T18:06:20.314000Z WriteTime: 2025-10-26T18:06:21.529000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:50340", "_ip": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:27.580002Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (20-20)
2025-10-26T18:06:27.580061Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:27.582224Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 69 offsets: 68 already_written: false write_statistics { persist_duration_ms: 278 queued_in_partition_duration_ms: 233 }
2025-10-26T18:06:27.582280Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 69
2025-10-26T18:06:27.583894Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 70 offsets: 69 already_written: false write_statistics { persist_duration_ms: 86 queued_in_partition_duration_ms: 405 }
2025-10-26T18:06:27.583945Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 70
2025-10-26T18:06:27.582663Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.582740Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.582859Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 54 size 8295660 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:27.582882Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:27.586641Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:27.596977Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 144  requestId: <none> cookie: 73
2025-10-26T18:06:27.597221Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.599164Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 77 to 77
2025-10-26T18:06:27.601193Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.601241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 77
2025-10-26T18:06:27.602976Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:27.614737Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:27.620502Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:27.621550Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.621582Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.622177Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 73 partNo : 0 messageNo: 145 size: 511959
2025-10-26T18:06:27.622693Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 73 partNo : 1 messageNo: 145 size: 511959
2025-10-26T18:06:27.624681Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 73 partNo : 2 messageNo: 145 size: 511959
2025-10-26T18:06:27.625386Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 73 partNo : 3 messageNo: 145 size: 511959
2025-10-26T18:06:27.625864Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 73 partNo : 4 messageNo: 145 size: 205040
2025-10-26T18:06:27.625907Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 73 partNo : 4 messageNo: 145 size 205040 offset: -1
2025-10-26T18:06:27.628405Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.628433Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.628589Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252911 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:27.628604Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.628627Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 71, partNo: 0, Offset: 70 is stored on disk
2025-10-26T18:06:27.628635Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.628653Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 71, partNo: 1, Offset: 70 is stored on disk
2025-10-26T18:06:27.628659Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.628674Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 71, partNo: 2, Offset: 70 is stored on disk
2025-10-26T18:06:27.628680Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.628695Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 71, partNo: 3, Offset: 70 is stored on disk
2025-10-26T18:06:27.628711Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.628727Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 71, partNo: 4, Offset: 70 is stored on disk
2025-10-26T18:06:27.634688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 78 to 78
2025-10-26T18:06:27.634381Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 72 partNo 0
2025-10-26T18:06:27.634434Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 72 partNo 1
2025-10-26T18:06:27.634451Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 72 partNo 2
2025-10-26T18:06:27.634470Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 72 partNo 3
2025-10-26T18:06:27.634488Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 72 partNo 4
2025-10-26T18:06:27.637678Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.637752Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 78
2025-10-26T18:06:27.640337Z :DEBUG: [/Root] Take Data. Partition 0. Read: {2, 0} (20-20)
2025-10-26T18:06:27.640407Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
2025-10-26T18:06:27.640439Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] In ContinueReadingDataImpl, ReadSizeBudget = 6758905, ReadSizeServerDelta = -6234617
2025-10-26T18:06:27.640792Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-26T18:06:27.640856Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 20 ProducerId: "src_id" SeqNo: 21 CreateTime: 2025-10-26T18:06:20.641000Z WriteTime: 2025-10-26T18:06:21.550000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:27.641906Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 1, data# { read_request { bytes_size: 6758905 } }
2025-10-26T18:06:27.642107Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 got read request: guid# 837e6991-c07f34a4-285ddd6d-c74f8ecf
2025-10-26T18:06:27.642160Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 performing read request: guid# 38fd5d14-bfcd4395-761c9705-919c80ac, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 46, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-26T18:06:27.642226Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 46 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 21 EndOffset 60 ClientCommitOffset 0 committedOffset 0 Guid 38fd5d14-bfcd4395-761c9705-919c80ac
2025-10-26T18:06:27.655913Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 72 partNo 4 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 2253230 count 1 nextOffset 72 batches 5
2025-10-26T18:06:27.656035Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 73 partNo 0
2025-10-26T18:06:27.659176Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 79 to 79
2025-10-26T18:06:27.665468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.665526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 79
2025-10-26T18:06:27.666561Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:27.687016Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 80 to 80
2025-10-26T18:06:27.689703Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.689743Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 80
2025-10-26T18:06:27.689770Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:27.690821Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:27.701827Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 73 partNo 0 result is x0000000000_00000000000000000068_00002_0000000004_00014 size 7988800
2025-10-26T18:06:27.705088Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 73 partNo 1
2025-10-26T18:06:27.705164Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 73 partNo 2
2025-10-26T18:06:27.705186Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 73 partNo 3
2025-10-26T18:06:27.705206Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 73 partNo 4
2025-10-26T18:06:27.705294Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000068_00002_0000000004_00014 new key d0000000000_00000000000000000068_00002_0000000004_00014 size 7988800 WTime 1761501987703
2025-10-26T18:06:27.743733Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 73 partNo 4 FormedBlobsCount 1 NewHead: Offset 72 PartNo 0 PackedSize 2253230 count 1 nextOffset 73 batches 5
2025-10-26T18:06:27.744725Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 72,1 HeadOffset 68 endOffset 71 curOffset 73 d0000000000_00000000000000000072_00000_0000000001_00004| size 2253220 WTime 1761501987744
2025-10-26T18:06:27.750270Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 146  requestId: <none> cookie: 74
2025-10-26T18:06:27.750401Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 141  requestId: <none> cookie: 71
2025-10-26T18:06:27.750509Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.750529Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.750593Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:27.751779Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.751861Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.768436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 71 offsets: 70 already_written: false write_statistics { persist_duration_ms: 97 queued_in_partition_duration_ms: 40 }
2025-10-26T18:06:27.768517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 71
2025-10-26T18:06:27.768558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:27.776113Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:27.783188Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:27.795356Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 81 to 81
2025-10-26T18:06:27.798774Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.798824Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.799380Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 74 partNo : 0 messageNo: 147 size: 511959
2025-10-26T18:06:27.799901Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 74 partNo : 1 messageNo: 147 size: 511959
2025-10-26T18:06:27.800461Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 74 partNo : 2 messageNo: 147 size: 511959
2025-10-26T18:06:27.800524Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 68 partNo 2 count 4 size 7988800
2025-10-26T18:06:27.802531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.802578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 81
2025-10-26T18:06:27.802607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:27.803584Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:27.803547Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 74 partNo : 3 messageNo: 147 size: 511959
2025-10-26T18:06:27.804268Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 74 partNo : 4 messageNo: 147 size: 205040
2025-10-26T18:06:27.804302Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 74 partNo : 4 messageNo: 147 size 205040 offset: -1
2025-10-26T18:06:27.808605Z node 2 :PERSQUEUE DEBUG: Read proxy: bootstrap for direct read id: 0
2025-10-26T18:06:27.816372Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.816420Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.816718Z node 2 :PERSQUEUE DEBUG: read cookie 10 Topic 'rt3.dc1--test-topic' partition 0 user user offset 21 count 46 size 524288 endOffset 71 max time lag 0ms effective offset 21
2025-10-26T18:06:27.816941Z node 2 :PERSQUEUE DEBUG: read cookie 10 added 2 blobs, size 7988782 count 4 last offset 25
2025-10-26T18:06:27.817014Z node 2 :PERSQUEUE DEBUG: Reading cookie 10. Send blob request.
2025-10-26T18:06:27.817385Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 148  requestId: <none> cookie: 75
2025-10-26T18:06:27.818888Z node 2 :PERSQUEUE DEBUG: Got data from cache. Partition 0 offset 18 count 3 source 0 size 8295638 accessed 0 times before, last time 2025-10-26T18:06:27.000000Z
2025-10-26T18:06:27.818910Z node 2 :PERSQUEUE DEBUG: No blob in L1. Partition 0 offset 21 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:27.818944Z node 2 :PERSQUEUE DEBUG: Reading cookie 10. Have to read 1 of 2 from KV
2025-10-26T18:06:27.817964Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.821328Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18
2025-10-26T18:06:27.821355Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21
2025-10-26T18:06:27.835958Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:27.838764Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.838803Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.839245Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 75 partNo : 0 messageNo: 149 size: 511959
2025-10-26T18:06:27.839644Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 75 partNo : 1 messageNo: 149 size: 511959
2025-10-26T18:06:27.840048Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 75 partNo : 2 messageNo: 149 size: 511959
2025-10-26T18:06:27.847903Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 75 partNo : 3 messageNo: 149 size: 511959
2025-10-26T18:06:27.848844Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 75 partNo : 4 messageNo: 149 size: 205040
2025-10-26T18:06:27.848873Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 75 partNo : 4 messageNo: 149 size 205040 offset: -1
2025-10-26T18:06:27.850223Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:27.853443Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.853481Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.853594Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 150  requestId: <none> cookie: 76
2025-10-26T18:06:27.857111Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.877115Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:27.879477Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.879553Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.880758Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 76 partNo : 0 messageNo: 151 size: 511959
2025-10-26T18:06:27.881403Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 76 partNo : 1 messageNo: 151 size: 511959
2025-10-26T18:06:27.881941Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 76 partNo : 2 messageNo: 151 size: 511959
2025-10-26T18:06:27.888736Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 76 partNo : 3 messageNo: 151 size: 511959
2025-10-26T18:06:27.888981Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 76 partNo : 4 messageNo: 151 size: 205040
2025-10-26T18:06:27.889012Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 76 partNo : 4 messageNo: 151 size 205040 offset: -1
2025-10-26T18:06:27.904725Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 68 count 4 size 7988800 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:27.904828Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 57 size 7988782
2025-10-26T18:06:27.904847Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 68 size 7988800
2025-10-26T18:06:27.904927Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505822 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:27.904946Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.904983Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 72, partNo: 0, Offset: 71 is stored on disk
2025-10-26T18:06:27.904996Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.905016Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 72, partNo: 1, Offset: 71 is stored on disk
2025-10-26T18:06:27.905034Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.905056Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 72, partNo: 2, Offset: 71 is stored on disk
2025-10-26T18:06:27.905066Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.905093Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 72, partNo: 3, Offset: 71 is stored on disk
2025-10-26T18:06:27.905113Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.905131Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 72, partNo: 4, Offset: 71 is stored on disk
2025-10-26T18:06:27.905139Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.905156Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 73, partNo: 0, Offset: 72 is stored on disk
2025-10-26T18:06:27.905164Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.905182Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 73, partNo: 1, Offset: 72 is stored on disk
2025-10-26T18:06:27.905190Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.905208Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 73, partNo: 2, Offset: 72 is stored on disk
2025-10-26T18:06:27.905216Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.905232Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 73, partNo: 3, Offset: 72 is stored on disk
2025-10-26T18:06:27.905242Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:27.905261Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 73, partNo: 4, Offset: 72 is stored on disk
2025-10-26T18:06:27.911258Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:27.913867Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 74 partNo 0
2025-10-26T18:06:27.913921Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 74 partNo 1
2025-10-26T18:06:27.913940Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 74 partNo 2
2025-10-26T18:06:27.913958Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 74 partNo 3
2025-10-26T18:06:27.913966Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 57 size 7988782 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:27.913974Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 74 partNo 4
2025-10-26T18:06:27.913985Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:27.931731Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 143  requestId: <none> cookie: 72
2025-10-26T18:06:27.931829Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 145  requestId: <none> cookie: 73
2025-10-26T18:06:27.931931Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:27.931961Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:27.933878Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.933979Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:27.934903Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 72 offsets: 71 already_written: false write_statistics { persist_duration_ms: 270 queued_in_partition_duration_ms: 37 }
2025-10-26T18:06:27.934959Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 72
2025-10-26T18:06:27.935012Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:27.936436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 73 offsets: 72 already_written: false write_statistics { persist_duration_ms: 201 queued_in_partition_duration_ms: 78 }
2025-10-26T18:06:27.936465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 73
2025-10-26T18:06:27.951603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 82 to 82
2025-10-26T18:06:27.958727Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:27.958772Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 82
2025-10-26T18:06:27.959310Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:27.947548Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 74 partNo 4 FormedBlobsCount 0 NewHead: Offset 73 PartNo 0 PackedSize 2253230 count 1 nextOffset 74 batches 5
2025-10-26T18:06:27.947682Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 75 partNo 0
2025-10-26T18:06:27.947716Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 75 partNo 1
2025-10-26T18:06:27.947735Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 75 partNo 2
2025-10-26T18:06:27.947751Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 75 partNo 3
2025-10-26T18:06:27.947767Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 75 partNo 4
2025-10-26T18:06:27.992587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 83 to 83
2025-10-26T18:06:27.994455Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 75 partNo 4 FormedBlobsCount 0 NewHead: Offset 73 PartNo 0 PackedSize 4506428 count 2 nextOffset 75 batches 9
2025-10-26T18:06:27.997807Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 76 partNo 0
2025-10-26T18:06:27.997874Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 76 partNo 1
2025-10-26T18:06:27.997893Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 76 partNo 2
2025-10-26T18:06:27.997911Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 76 partNo 3
2025-10-26T18:06:28.004184Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:28.004234Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 83
2025-10-26T18:06:28.004264Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:28.004823Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:28.042120Z node 2 :PERSQUEUE DEBUG: Got results. 1 of 2 from KV. Status 1
2025-10-26T18:06:28.042163Z node 2 :PERSQUEUE DEBUG: Got results. result 0 from KV. Status 0
2025-10-26T18:06:28.042205Z node 2 :PERSQUEUE DEBUG: Prefetched blob in L1. Partition 0 offset 21 count 4 size 7988782 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:28.042495Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 61 size 8295582
2025-10-26T18:06:28.042563Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 size 7988782
2025-10-26T18:06:28.058886Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 61 size 8295582 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:28.058915Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:28.063197Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 76 partNo 3 result is x0000000000_00000000000000000072_00000_0000000003_00015 size 8295660
2025-10-26T18:06:28.077626Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 76 partNo 4
2025-10-26T18:06:28.077915Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000072_00000_0000000003_00015 new key d0000000000_00000000000000000072_00000_0000000003_00015 size 8295660 WTime 1761501988077
2025-10-26T18:06:28.082721Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 76 partNo 4 FormedBlobsCount 1 NewHead: Offset 75 PartNo 3 PackedSize 717145 count 1 nextOffset 76 batches 2
2025-10-26T18:06:28.083729Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 75,1 HeadOffset 72 endOffset 73 curOffset 76 d0000000000_00000000000000000075_00003_0000000001_00001| size 717135 WTime 1761501988083
2025-10-26T18:06:28.085380Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:28.085447Z node 2 :PERSQUEUE DEBUG: FormAnswer 2
2025-10-26T18:06:28.107316Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 72 partNo 0 count 3 size 8295660
2025-10-26T18:06:28.112667Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 152  requestId: <none> cookie: 77
2025-10-26T18:06:28.113016Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.116430Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 21 totakecount 3 count 0 size 1536000 from pos 0 cbcount 3
2025-10-26T18:06:28.121267Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 21 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:28.125339Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:28.131546Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 21 totakecount 4 count 1 size 717095 from pos 0 cbcount 2
2025-10-26T18:06:28.133642Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 22 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:28.144003Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.144053Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.144775Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 77 partNo : 0 messageNo: 153 size: 511959
2025-10-26T18:06:28.145317Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 77 partNo : 1 messageNo: 153 size: 511959
2025-10-26T18:06:28.148034Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 22 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:28.149611Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 22 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:28.154016Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 77 partNo : 2 messageNo: 153 size: 511959
2025-10-26T18:06:28.154730Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 77 partNo : 3 messageNo: 153 size: 511959
2025-10-26T18:06:28.155798Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 77 partNo : 4 messageNo: 153 size: 205040
2025-10-26T18:06:28.155836Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 77 partNo : 4 messageNo: 153 size 205040 offset: -1
2025-10-26T18:06:28.162433Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 22 totakecount 4 count 1 size 205086 from pos 0 cbcount 1
2025-10-26T18:06:28.164481Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 23 totakecount 4 count 0 size 512009 from pos 0 cbcount 1
2025-10-26T18:06:28.173032Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:28.174934Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 23 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:28.175378Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.175414Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.175530Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 72 count 3 size 8295660 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:28.175637Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 64 size 7988760
2025-10-26T18:06:28.175688Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 72 size 8295660
2025-10-26T18:06:28.175753Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 64 size 7988760 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:28.175775Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:28.181318Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 23 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:28.183113Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 23 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:28.185403Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 23 totakecount 4 count 1 size 717095 from pos 0 cbcount 2
2025-10-26T18:06:28.187580Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 24 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:28.189437Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 24 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:28.191153Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 24 totakecount 4 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:28.192270Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 24 totakecount 4 count 1 size 205086 from pos 0 cbcount 1
2025-10-26T18:06:28.193476Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 25 totakecount 4 count 0 size 512009 from pos 0 cbcount 1
2025-10-26T18:06:28.194492Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 6758733 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:28.194521Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194555Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 74, partNo: 0, Offset: 73 is stored on disk
2025-10-26T18:06:28.194566Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194588Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 74, partNo: 1, Offset: 73 is stored on disk
2025-10-26T18:06:28.194595Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194615Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 74, partNo: 2, Offset: 73 is stored on disk
2025-10-26T18:06:28.194623Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194644Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 74, partNo: 3, Offset: 73 is stored on disk
2025-10-26T18:06:28.194664Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194683Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 74, partNo: 4, Offset: 73 is stored on disk
2025-10-26T18:06:28.194691Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194710Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 75, partNo: 0, Offset: 74 is stored on disk
2025-10-26T18:06:28.194717Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194739Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 75, partNo: 1, Offset: 74 is stored on disk
2025-10-26T18:06:28.194748Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194768Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 75, partNo: 2, Offset: 74 is stored on disk
2025-10-26T18:06:28.194779Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194797Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 75, partNo: 3, Offset: 74 is stored on disk
2025-10-26T18:06:28.194809Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194829Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 75, partNo: 4, Offset: 74 is stored on disk
2025-10-26T18:06:28.194837Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194857Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 76, partNo: 0, Offset: 75 is stored on disk
2025-10-26T18:06:28.194865Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194888Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 76, partNo: 1, Offset: 75 is stored on disk
2025-10-26T18:06:28.194895Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194913Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 76, partNo: 2, Offset: 75 is stored on disk
2025-10-26T18:06:28.194922Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194956Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 76, partNo: 3, Offset: 75 is stored on disk
2025-10-26T18:06:28.194970Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.194987Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 76, partNo: 4, Offset: 75 is stored on disk
2025-10-26T18:06:28.197254Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 77 partNo 0
2025-10-26T18:06:28.197300Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 77 partNo 1
2025-10-26T18:06:28.197316Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 77 partNo 2
2025-10-26T18:06:28.197334Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 77 partNo 3
2025-10-26T18:06:28.197350Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 77 partNo 4
2025-10-26T18:06:28.211484Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 21
2025-10-26T18:06:28.211956Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 154  requestId: <none> cookie: 78
2025-10-26T18:06:28.212052Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 147  requestId: <none> cookie: 74
2025-10-26T18:06:28.212118Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 149  requestId: <none> cookie: 75
2025-10-26T18:06:28.212181Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 151  requestId: <none> cookie: 76
2025-10-26T18:06:28.212278Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.212394Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.212448Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.222870Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.233426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 74 offsets: 73 already_written: false write_statistics { persist_duration_ms: 280 queued_in_partition_duration_ms: 97 }
2025-10-26T18:06:28.233942Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 74
2025-10-26T18:06:28.233985Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:28.243853Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 77 partNo 4 FormedBlobsCount 0 NewHead: Offset 76 PartNo 0 PackedSize 2253230 count 1 nextOffset 77 batches 5
2025-10-26T18:06:28.250147Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 76,1 HeadOffset 75 endOffset 76 curOffset 77 d0000000000_00000000000000000076_00000_0000000001_00004| size 2253220 WTime 1761501988249
2025-10-26T18:06:28.260126Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 75 offsets: 74 already_written: false write_statistics { persist_duration_ms: 246 queued_in_partition_duration_ms: 97 }
2025-10-26T18:06:28.260176Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 75
2025-10-26T18:06:28.263874Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 76 offsets: 75 already_written: false write_statistics { persist_duration_ms: 196 queued_in_partition_duration_ms: 111 }
2025-10-26T18:06:28.263918Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 76
2025-10-26T18:06:28.279490Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:28.282461Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.282513Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.283062Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 78 partNo : 0 messageNo: 155 size: 511959
2025-10-26T18:06:28.283774Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 78 partNo : 1 messageNo: 155 size: 511959
2025-10-26T18:06:28.286473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 84 to 84
2025-10-26T18:06:28.292136Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 78 partNo : 2 messageNo: 155 size: 511959
2025-10-26T18:06:28.292619Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 78 partNo : 3 messageNo: 155 size: 511959
2025-10-26T18:06:28.292824Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 78 partNo : 4 messageNo: 155 size: 205040
2025-10-26T18:06:28.292840Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 78 partNo : 4 messageNo: 155 size 205040 offset: -1
2025-10-26T18:06:28.295298Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:28.302764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:28.302819Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 84
2025-10-26T18:06:28.303809Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:28.326720Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:28.327201Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.327243Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.327333Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 156  requestId: <none> cookie: 79
2025-10-26T18:06:28.327679Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.349048Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.349097Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.349655Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 79 partNo : 0 messageNo: 157 size: 511959
2025-10-26T18:06:28.351871Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:28.352900Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 79 partNo : 1 messageNo: 157 size: 511959
2025-10-26T18:06:28.353383Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 79 partNo : 2 messageNo: 157 size: 511959
2025-10-26T18:06:28.353873Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 79 partNo : 3 messageNo: 157 size: 511959
2025-10-26T18:06:28.354101Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 79 partNo : 4 messageNo: 157 size: 205040
2025-10-26T18:06:28.354126Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 79 partNo : 4 messageNo: 157 size 205040 offset: -1
2025-10-26T18:06:28.374376Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:28.385728Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 85 to 85
2025-10-26T18:06:28.388590Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.388637Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.389030Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252911 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:28.389055Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.389090Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 77, partNo: 0, Offset: 76 is stored on disk
2025-10-26T18:06:28.389101Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.389122Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 77, partNo: 1, Offset: 76 is stored on disk
2025-10-26T18:06:28.389130Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.389150Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 77, partNo: 2, Offset: 76 is stored on disk
2025-10-26T18:06:28.389158Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.389179Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 77, partNo: 3, Offset: 76 is stored on disk
2025-10-26T18:06:28.389201Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.389222Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 77, partNo: 4, Offset: 76 is stored on disk
2025-10-26T18:06:28.393126Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:28.393172Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 85
2025-10-26T18:06:28.393831Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:28.422358Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 78 partNo 0
2025-10-26T18:06:28.422450Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 78 partNo 1
2025-10-26T18:06:28.422487Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 78 partNo 2
2025-10-26T18:06:28.422522Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 78 partNo 3
2025-10-26T18:06:28.422556Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 78 partNo 4
2025-10-26T18:06:28.434761Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 158  requestId: <none> cookie: 80
2025-10-26T18:06:28.434894Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 153  requestId: <none> cookie: 77
2025-10-26T18:06:28.451870Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.451962Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.468135Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 86 to 86
2025-10-26T18:06:28.487194Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:28.487245Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 86
2025-10-26T18:06:28.487276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:28.488768Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:28.506122Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 77 offsets: 76 already_written: false write_statistics { persist_duration_ms: 191 queued_in_partition_duration_ms: 2 }
2025-10-26T18:06:28.506175Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 77
2025-10-26T18:06:28.506210Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:28.511111Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 78 partNo 4 FormedBlobsCount 0 NewHead: Offset 77 PartNo 0 PackedSize 2253230 count 1 nextOffset 78 batches 5
2025-10-26T18:06:28.511326Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 79 partNo 0
2025-10-26T18:06:28.511383Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 79 partNo 1
2025-10-26T18:06:28.511431Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 79 partNo 2
2025-10-26T18:06:28.511474Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 79 partNo 3
2025-10-26T18:06:28.511515Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 79 partNo 4
2025-10-26T18:06:28.515854Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 73 Result { Offset: 21 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 22 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981574 CreateTimestampMS: 1761501980917 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 22 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 23 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981638 CreateTimestampMS: 1761501981002 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 23 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 24 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981851 CreateTimestampMS: 1761501981070 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 24 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 25 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981906 CreateTimestampMS: 1761501981417 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 1 BlobsFromCache: 1 SizeLag: 114708361 RealReadOffset: 24 WaitQuotaTimeMs: 0 } Cookie: 21 }
2025-10-26T18:06:28.521290Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.521346Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.521959Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 80 partNo : 0 messageNo: 159 size: 511959
2025-10-26T18:06:28.522525Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 80 partNo : 1 messageNo: 159 size: 511959
2025-10-26T18:06:28.523093Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 80 partNo : 2 messageNo: 159 size: 511959
2025-10-26T18:06:28.523582Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 80 partNo : 3 messageNo: 159 size: 511959
2025-10-26T18:06:28.523824Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 80 partNo : 4 messageNo: 159 size: 205040
2025-10-26T18:06:28.523846Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 80 partNo : 4 messageNo: 159 size 205040 offset: -1
2025-10-26T18:06:28.541795Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:28.557486Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 79 partNo 4 FormedBlobsCount 0 NewHead: Offset 77 PartNo 0 PackedSize 4506428 count 2 nextOffset 79 batches 9
2025-10-26T18:06:28.562634Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 77,2 HeadOffset 75 endOffset 77 curOffset 79 d0000000000_00000000000000000077_00000_0000000002_00008| size 4506418 WTime 1761501988561
2025-10-26T18:06:28.576912Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:28.586448Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.586039Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:28.586731Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.586488Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.586567Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 160  requestId: <none> cookie: 81
2025-10-26T18:06:28.593013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 87 to 87
2025-10-26T18:06:28.595034Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:28.595079Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 87
2025-10-26T18:06:28.595108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:28.595625Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:28.605814Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:28.619008Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.619056Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.619624Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 81 partNo : 0 messageNo: 161 size: 511959
2025-10-26T18:06:28.620154Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 81 partNo : 1 messageNo: 161 size: 511959
2025-10-26T18:06:28.620703Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 81 partNo : 2 messageNo: 161 size: 511959
2025-10-26T18:06:28.621230Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 81 partNo : 3 messageNo: 161 size: 511959
2025-10-26T18:06:28.621482Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 81 partNo : 4 messageNo: 161 size: 205040
2025-10-26T18:06:28.621520Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 81 partNo : 4 messageNo: 161 size 205040 offset: -1
2025-10-26T18:06:28.632954Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:28.637228Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 25 endOffset 73
2025-10-26T18:06:28.637285Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 73 ReadOffset 25 ReadGuid 38fd5d14-bfcd4395-761c9705-919c80ac has messages 1
2025-10-26T18:06:28.637769Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.637803Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.637911Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 162  requestId: <none> cookie: 82
2025-10-26T18:06:28.641166Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 25, endOffset# 73, WTime# 1761501981906, sizeLag# 114708361
2025-10-26T18:06:28.641198Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1TEvPartitionReady. Aval parts: 0
2025-10-26T18:06:28.641271Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 read done: guid# 38fd5d14-bfcd4395-761c9705-919c80ac, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 9011865
2025-10-26T18:06:28.641309Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 response to read: guid# 38fd5d14-bfcd4395-761c9705-919c80ac
2025-10-26T18:06:28.652905Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505822 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:28.652949Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.652990Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 78, partNo: 0, Offset: 77 is stored on disk
2025-10-26T18:06:28.653003Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.653027Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 78, partNo: 1, Offset: 77 is stored on disk
2025-10-26T18:06:28.653035Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.653055Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 78, partNo: 2, Offset: 77 is stored on disk
2025-10-26T18:06:28.653064Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.653086Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 78, partNo: 3, Offset: 77 is stored on disk
2025-10-26T18:06:28.653106Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.653127Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 78, partNo: 4, Offset: 77 is stored on disk
2025-10-26T18:06:28.653136Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.653156Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 79, partNo: 0, Offset: 78 is stored on disk
2025-10-26T18:06:28.653163Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.653183Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 79, partNo: 1, Offset: 78 is stored on disk
2025-10-26T18:06:28.653196Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.653217Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 79, partNo: 2, Offset: 78 is stored on disk
2025-10-26T18:06:28.653225Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.653245Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 79, partNo: 3, Offset: 78 is stored on disk
2025-10-26T18:06:28.653257Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.653277Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 79, partNo: 4, Offset: 78 is stored on disk
2025-10-26T18:06:28.654844Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 155  requestId: <none> cookie: 78
2025-10-26T18:06:28.654920Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 157  requestId: <none> cookie: 79
2025-10-26T18:06:28.660788Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.660884Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.660932Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.670980Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.671030Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.671567Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 82 partNo : 0 messageNo: 163 size: 511959
2025-10-26T18:06:28.672066Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 82 partNo : 1 messageNo: 163 size: 511959
2025-10-26T18:06:28.672576Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 82 partNo : 2 messageNo: 163 size: 511959
2025-10-26T18:06:28.673103Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 82 partNo : 3 messageNo: 163 size: 511959
2025-10-26T18:06:28.673355Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 82 partNo : 4 messageNo: 163 size: 205040
2025-10-26T18:06:28.673377Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 82 partNo : 4 messageNo: 163 size 205040 offset: -1
2025-10-26T18:06:28.682546Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 80 partNo 0
2025-10-26T18:06:28.682603Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 80 partNo 1
2025-10-26T18:06:28.690387Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 78 offsets: 77 already_written: false write_statistics { persist_duration_ms: 230 queued_in_partition_duration_ms: 97 }
2025-10-26T18:06:28.690457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 78
2025-10-26T18:06:28.690483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:28.701804Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:28.717292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 88 to 88
2025-10-26T18:06:28.719585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:28.719622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 88
2025-10-26T18:06:28.719643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:28.720236Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 79 offsets: 78 already_written: false write_statistics { persist_duration_ms: 141 queued_in_partition_duration_ms: 149 }
2025-10-26T18:06:28.720276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 79
2025-10-26T18:06:28.720302Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:28.721820Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:28.728683Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Process answer. Aval parts: 1
2025-10-26T18:06:28.743912Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:28.744410Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.744455Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.751259Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 89 to 89
2025-10-26T18:06:28.753582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:28.753630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 89
2025-10-26T18:06:28.753659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:28.754232Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:28.776735Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 80 partNo 1 result is x0000000000_00000000000000000075_00003_0000000004_00014 size 7988804
2025-10-26T18:06:28.783229Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Got ReadResponse, serverBytesSize = 9011865, now ReadSizeBudget = 0, ReadSizeServerDelta = -8487577
2025-10-26T18:06:28.785805Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (21-21)
2025-10-26T18:06:28.785863Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 9011865 to budget
2025-10-26T18:06:28.789577Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 80 partNo 2
2025-10-26T18:06:28.790171Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 80 partNo 3
2025-10-26T18:06:28.790204Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 80 partNo 4
2025-10-26T18:06:28.790271Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000075_00003_0000000004_00014 new key d0000000000_00000000000000000075_00003_0000000004_00014 size 7988804 WTime 1761501988789
2025-10-26T18:06:28.801611Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 80 partNo 4 FormedBlobsCount 1 NewHead: Offset 79 PartNo 1 PackedSize 1741199 count 1 nextOffset 80 batches 4
2025-10-26T18:06:28.809251Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 81 partNo 0
2025-10-26T18:06:28.809304Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 81 partNo 1
2025-10-26T18:06:28.809322Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 81 partNo 2
2025-10-26T18:06:28.809337Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 81 partNo 3
2025-10-26T18:06:28.809354Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 81 partNo 4
2025-10-26T18:06:28.825061Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 81 partNo 4 FormedBlobsCount 0 NewHead: Offset 79 PartNo 1 PackedSize 3994397 count 2 nextOffset 81 batches 8
2025-10-26T18:06:28.826093Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 79,2 HeadOffset 75 endOffset 79 curOffset 81 d0000000000_00000000000000000079_00001_0000000002_00007| size 3994387 WTime 1761501988825
2025-10-26T18:06:28.833913Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:28.837146Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 164  requestId: <none> cookie: 83
2025-10-26T18:06:28.837487Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.845010Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:28.854359Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:28.855835Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.855879Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.856388Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 83 partNo : 0 messageNo: 165 size: 511959
2025-10-26T18:06:28.856864Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 83 partNo : 1 messageNo: 165 size: 511959
2025-10-26T18:06:28.857349Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 83 partNo : 2 messageNo: 165 size: 511959
2025-10-26T18:06:28.859552Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 83 partNo : 3 messageNo: 165 size: 511959
2025-10-26T18:06:28.859794Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 83 partNo : 4 messageNo: 165 size: 205040
2025-10-26T18:06:28.859818Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 83 partNo : 4 messageNo: 165 size 205040 offset: -1
2025-10-26T18:06:28.864198Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.864233Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.864294Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 166  requestId: <none> cookie: 84
2025-10-26T18:06:28.864697Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.865821Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 75 partNo 3 count 4 size 7988804
2025-10-26T18:06:28.886936Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.886990Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.887568Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 84 partNo : 0 messageNo: 167 size: 511959
2025-10-26T18:06:28.888129Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 84 partNo : 1 messageNo: 167 size: 511959
2025-10-26T18:06:28.888561Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 84 partNo : 2 messageNo: 167 size: 511959
2025-10-26T18:06:28.888971Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 84 partNo : 3 messageNo: 167 size: 511959
2025-10-26T18:06:28.889158Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 84 partNo : 4 messageNo: 167 size: 205040
2025-10-26T18:06:28.889173Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 84 partNo : 4 messageNo: 167 size 205040 offset: -1
2025-10-26T18:06:28.892574Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:28.897292Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:28.897665Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.897714Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.897840Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 168  requestId: <none> cookie: 85
2025-10-26T18:06:28.898243Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.920400Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.920447Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.928468Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 85 partNo : 0 messageNo: 169 size: 511959
2025-10-26T18:06:28.930959Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 85 partNo : 1 messageNo: 169 size: 511959
2025-10-26T18:06:28.932996Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 85 partNo : 2 messageNo: 169 size: 511959
2025-10-26T18:06:28.933480Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 85 partNo : 3 messageNo: 169 size: 511959
2025-10-26T18:06:28.933709Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 85 partNo : 4 messageNo: 169 size: 205040
2025-10-26T18:06:28.933730Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 85 partNo : 4 messageNo: 169 size 205040 offset: -1
2025-10-26T18:06:28.941105Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:28.963845Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:28.965837Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:28.965880Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:28.965973Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 170  requestId: <none> cookie: 86
2025-10-26T18:06:28.969390Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:28.971761Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 75 count 4 size 7988804 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:28.971896Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505822 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:28.971929Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.971946Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 18 size 8295638
2025-10-26T18:06:28.971968Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 80, partNo: 0, Offset: 79 is stored on disk
2025-10-26T18:06:28.971970Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 75 size 7988804
2025-10-26T18:06:28.971983Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.972010Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 80, partNo: 1, Offset: 79 is stored on disk
2025-10-26T18:06:28.972024Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.972045Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 80, partNo: 2, Offset: 79 is stored on disk
2025-10-26T18:06:28.972057Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.972080Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 80, partNo: 3, Offset: 79 is stored on disk
2025-10-26T18:06:28.972106Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.972127Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 80, partNo: 4, Offset: 79 is stored on disk
2025-10-26T18:06:28.972136Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.972155Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 81, partNo: 0, Offset: 80 is stored on disk
2025-10-26T18:06:28.972163Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.972180Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 81, partNo: 1, Offset: 80 is stored on disk
2025-10-26T18:06:28.972188Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.972215Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 81, partNo: 2, Offset: 80 is stored on disk
2025-10-26T18:06:28.972224Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.972240Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 81, partNo: 3, Offset: 80 is stored on disk
2025-10-26T18:06:28.972252Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:28.972270Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 81, partNo: 4, Offset: 80 is stored on disk
2025-10-26T18:06:28.994809Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 82 partNo 0
2025-10-26T18:06:28.994859Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 82 partNo 1
2025-10-26T18:06:28.994877Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 82 partNo 2
2025-10-26T18:06:28.994905Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 82 partNo 3
2025-10-26T18:06:28.994922Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 82 partNo 4
2025-10-26T18:06:29.006415Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 18 size 8295638 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:29.006448Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:29.009095Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 159  requestId: <none> cookie: 80
2025-10-26T18:06:29.009182Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 161  requestId: <none> cookie: 81
2025-10-26T18:06:29.009524Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.028341Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 80 offsets: 79 already_written: false write_statistics { persist_duration_ms: 289 queued_in_partition_duration_ms: 105 }
2025-10-26T18:06:29.028386Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 80
2025-10-26T18:06:29.028412Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:29.038203Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.038242Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.038712Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 86 partNo : 0 messageNo: 171 size: 511959
2025-10-26T18:06:29.039204Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 86 partNo : 1 messageNo: 171 size: 511959
2025-10-26T18:06:29.039626Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 86 partNo : 2 messageNo: 171 size: 511959
2025-10-26T18:06:29.040006Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 86 partNo : 3 messageNo: 171 size: 511959
2025-10-26T18:06:29.040311Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 86 partNo : 4 messageNo: 171 size: 205040
2025-10-26T18:06:29.040324Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 86 partNo : 4 messageNo: 171 size 205040 offset: -1
2025-10-26T18:06:29.068126Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 90 to 90
2025-10-26T18:06:29.088620Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.092549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:29.092587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 90
2025-10-26T18:06:29.092608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:29.093297Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (21-21)
2025-10-26T18:06:29.093336Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 21 ProducerId: "src_id" SeqNo: 22 CreateTime: 2025-10-26T18:06:20.917000Z WriteTime: 2025-10-26T18:06:21.574000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:29.093831Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (22-22)
2025-10-26T18:06:29.095617Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:29.093866Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:29.105398Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:29.106028Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 81 offsets: 80 already_written: false write_statistics { persist_duration_ms: 162 queued_in_partition_duration_ms: 183 }
2025-10-26T18:06:29.113859Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 81
2025-10-26T18:06:29.113932Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:29.154208Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 82 partNo 4 FormedBlobsCount 0 NewHead: Offset 81 PartNo 0 PackedSize 2253230 count 1 nextOffset 82 batches 5
2025-10-26T18:06:29.154336Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 83 partNo 0
2025-10-26T18:06:29.154371Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 83 partNo 1
2025-10-26T18:06:29.154390Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 83 partNo 2
2025-10-26T18:06:29.154406Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 83 partNo 3
2025-10-26T18:06:29.154422Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 83 partNo 4
2025-10-26T18:06:29.166926Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 91 to 91
2025-10-26T18:06:29.189838Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:29.189882Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 91
2025-10-26T18:06:29.189912Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:29.190782Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:29.225193Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 83 partNo 4 result is x0000000000_00000000000000000079_00001_0000000003_00015 size 8295604
2025-10-26T18:06:29.225293Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000079_00001_0000000003_00015 new key d0000000000_00000000000000000079_00001_0000000003_00015 size 8295604 WTime 1761501989225
2025-10-26T18:06:29.227628Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 83 partNo 4 FormedBlobsCount 1 NewHead: Offset 82 PartNo 4 PackedSize 205118 count 1 nextOffset 83 batches 1
2025-10-26T18:06:29.247600Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 84 partNo 0
2025-10-26T18:06:29.247672Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 84 partNo 1
2025-10-26T18:06:29.247690Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 84 partNo 2
2025-10-26T18:06:29.247709Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 84 partNo 3
2025-10-26T18:06:29.247726Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 84 partNo 4
2025-10-26T18:06:29.264859Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 84 partNo 4 FormedBlobsCount 0 NewHead: Offset 82 PartNo 4 PackedSize 2458316 count 2 nextOffset 84 batches 5
2025-10-26T18:06:29.265011Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 85 partNo 0
2025-10-26T18:06:29.265049Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 85 partNo 1
2025-10-26T18:06:29.265069Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 85 partNo 2
2025-10-26T18:06:29.265088Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 85 partNo 3
2025-10-26T18:06:29.265108Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 85 partNo 4
2025-10-26T18:06:29.279437Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 85 partNo 4 FormedBlobsCount 0 NewHead: Offset 82 PartNo 4 PackedSize 4711514 count 3 nextOffset 85 batches 9
2025-10-26T18:06:29.280626Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 82,3 HeadOffset 79 endOffset 81 curOffset 85 d0000000000_00000000000000000082_00004_0000000003_00008| size 4711504 WTime 1761501989280
2025-10-26T18:06:29.290640Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.290678Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.290782Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:29.309290Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 79 partNo 1 count 3 size 8295604
2025-10-26T18:06:29.334038Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 172  requestId: <none> cookie: 87
2025-10-26T18:06:29.334713Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.343989Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:29.345108Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.345161Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.345841Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 87 partNo : 0 messageNo: 173 size: 511959
2025-10-26T18:06:29.346418Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 87 partNo : 1 messageNo: 173 size: 511959
2025-10-26T18:06:29.346963Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 87 partNo : 2 messageNo: 173 size: 511959
2025-10-26T18:06:29.347591Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 87 partNo : 3 messageNo: 173 size: 511959
2025-10-26T18:06:29.348864Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 87 partNo : 4 messageNo: 173 size: 205040
2025-10-26T18:06:29.348904Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 87 partNo : 4 messageNo: 173 size 205040 offset: -1
2025-10-26T18:06:29.352143Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:29.352683Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.352727Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.352821Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 174  requestId: <none> cookie: 88
2025-10-26T18:06:29.353173Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.366211Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:29.367896Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.367939Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.368387Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 88 partNo : 0 messageNo: 175 size: 511959
2025-10-26T18:06:29.368891Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 88 partNo : 1 messageNo: 175 size: 511959
2025-10-26T18:06:29.369398Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 88 partNo : 2 messageNo: 175 size: 511959
2025-10-26T18:06:29.371830Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 88 partNo : 3 messageNo: 175 size: 511959
2025-10-26T18:06:29.372121Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 88 partNo : 4 messageNo: 175 size: 205040
2025-10-26T18:06:29.372144Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 88 partNo : 4 messageNo: 175 size 205040 offset: -1
2025-10-26T18:06:29.382939Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 79 count 3 size 8295604 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:29.383084Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 68 size 7988800
2025-10-26T18:06:29.383109Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 79 size 8295604
2025-10-26T18:06:29.383298Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 9011644 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:29.383318Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383356Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 82, partNo: 0, Offset: 81 is stored on disk
2025-10-26T18:06:29.383368Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383386Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 82, partNo: 1, Offset: 81 is stored on disk
2025-10-26T18:06:29.383393Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383411Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 82, partNo: 2, Offset: 81 is stored on disk
2025-10-26T18:06:29.383419Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383436Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 82, partNo: 3, Offset: 81 is stored on disk
2025-10-26T18:06:29.383455Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383473Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 82, partNo: 4, Offset: 81 is stored on disk
2025-10-26T18:06:29.383482Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383500Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 83, partNo: 0, Offset: 82 is stored on disk
2025-10-26T18:06:29.383507Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383524Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 83, partNo: 1, Offset: 82 is stored on disk
2025-10-26T18:06:29.383532Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383553Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 83, partNo: 2, Offset: 82 is stored on disk
2025-10-26T18:06:29.383559Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383576Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 83, partNo: 3, Offset: 82 is stored on disk
2025-10-26T18:06:29.383588Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383605Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 83, partNo: 4, Offset: 82 is stored on disk
2025-10-26T18:06:29.383612Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383629Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 84, partNo: 0, Offset: 83 is stored on disk
2025-10-26T18:06:29.383637Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383654Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 84, partNo: 1, Offset: 83 is stored on disk
2025-10-26T18:06:29.383661Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383677Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 84, partNo: 2, Offset: 83 is stored on disk
2025-10-26T18:06:29.383684Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383700Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 84, partNo: 3, Offset: 83 is stored on disk
2025-10-26T18:06:29.383710Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383729Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 84, partNo: 4, Offset: 83 is stored on disk
2025-10-26T18:06:29.383736Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383753Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 85, partNo: 0, Offset: 84 is stored on disk
2025-10-26T18:06:29.383759Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383777Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 85, partNo: 1, Offset: 84 is stored on disk
2025-10-26T18:06:29.383783Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383800Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 85, partNo: 2, Offset: 84 is stored on disk
2025-10-26T18:06:29.383807Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383823Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 85, partNo: 3, Offset: 84 is stored on disk
2025-10-26T18:06:29.383833Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.383850Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 85, partNo: 4, Offset: 84 is stored on disk
2025-10-26T18:06:29.384814Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 163  requestId: <none> cookie: 82
2025-10-26T18:06:29.384977Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 165  requestId: <none> cookie: 83
2025-10-26T18:06:29.385114Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 167  requestId: <none> cookie: 84
2025-10-26T18:06:29.385254Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 169  requestId: <none> cookie: 85
2025-10-26T18:06:29.385394Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 68 size 7988800 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:29.385415Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:29.396735Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 82 offsets: 81 already_written: false write_statistics { persist_duration_ms: 388 queued_in_partition_duration_ms: 161 }
2025-10-26T18:06:29.396816Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 82
2025-10-26T18:06:29.396853Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:29.393889Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.393982Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.394018Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.398215Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 83 offsets: 82 already_written: false write_statistics { persist_duration_ms: 228 queued_in_partition_duration_ms: 290 }
2025-10-26T18:06:29.398257Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 83
2025-10-26T18:06:29.394051Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.394133Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:29.409205Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.409251Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.416640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 92 to 92
2025-10-26T18:06:29.417659Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 86 partNo 0
2025-10-26T18:06:29.417738Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 86 partNo 1
2025-10-26T18:06:29.417766Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 86 partNo 2
2025-10-26T18:06:29.417793Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 86 partNo 3
2025-10-26T18:06:29.417810Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 86 partNo 4
2025-10-26T18:06:29.424291Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:29.424342Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 92
2025-10-26T18:06:29.425212Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (22-22)
2025-10-26T18:06:29.425264Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 22 ProducerId: "src_id" SeqNo: 23 CreateTime: 2025-10-26T18:06:21.002000Z WriteTime: 2025-10-26T18:06:21.638000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:29.427480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 84 offsets: 83 already_written: false write_statistics { persist_duration_ms: 135 queued_in_partition_duration_ms: 358 }
2025-10-26T18:06:29.427530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 84
2025-10-26T18:06:29.428936Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 85 offsets: 84 already_written: false write_statistics { persist_duration_ms: 118 queued_in_partition_duration_ms: 331 }
2025-10-26T18:06:29.428965Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 85
2025-10-26T18:06:29.429808Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (23-23)
2025-10-26T18:06:29.429898Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:29.437080Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 93 to 93
2025-10-26T18:06:29.439340Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:29.439382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 93
2025-10-26T18:06:29.440291Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:29.452756Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 94 to 94
2025-10-26T18:06:29.460691Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:29.460733Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 94
2025-10-26T18:06:29.461098Z :DEBUG: [/Root] Take Data. Partition 0. Read: {2, 0} (23-23)
2025-10-26T18:06:29.461136Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 23 ProducerId: "src_id" SeqNo: 24 CreateTime: 2025-10-26T18:06:21.070000Z WriteTime: 2025-10-26T18:06:21.851000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:50340", "_ip": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:29.465542Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (24-24)
2025-10-26T18:06:29.465593Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
2025-10-26T18:06:29.462460Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 86 partNo 4 FormedBlobsCount 0 NewHead: Offset 85 PartNo 0 PackedSize 2253230 count 1 nextOffset 86 batches 5
2025-10-26T18:06:29.462602Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 87 partNo 0
2025-10-26T18:06:29.462636Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 87 partNo 1
2025-10-26T18:06:29.462656Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 87 partNo 2
2025-10-26T18:06:29.475017Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 95 to 95
2025-10-26T18:06:29.477185Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:29.477233Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 95
2025-10-26T18:06:29.477264Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:29.481586Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:29.502695Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 87 partNo 2 result is x0000000000_00000000000000000082_00004_0000000004_00014 size 7988760
2025-10-26T18:06:29.504663Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 87 partNo 3
2025-10-26T18:06:29.504707Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 87 partNo 4
2025-10-26T18:06:29.504775Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000082_00004_0000000004_00014 new key d0000000000_00000000000000000082_00004_0000000004_00014 size 7988760 WTime 1761501989504
2025-10-26T18:06:29.525479Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 87 partNo 4 FormedBlobsCount 1 NewHead: Offset 86 PartNo 2 PackedSize 1229172 count 1 nextOffset 87 batches 3
2025-10-26T18:06:29.525590Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 88 partNo 0
2025-10-26T18:06:29.525625Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 88 partNo 1
2025-10-26T18:06:29.525643Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 88 partNo 2
2025-10-26T18:06:29.525661Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 88 partNo 3
2025-10-26T18:06:29.525678Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 88 partNo 4
2025-10-26T18:06:29.548200Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 88 partNo 4 FormedBlobsCount 0 NewHead: Offset 86 PartNo 2 PackedSize 3482370 count 2 nextOffset 88 batches 7
2025-10-26T18:06:29.549154Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 86,2 HeadOffset 82 endOffset 85 curOffset 88 d0000000000_00000000000000000086_00002_0000000002_00006| size 3482360 WTime 1761501989548
2025-10-26T18:06:29.556160Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:29.556259Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 176  requestId: <none> cookie: 89
2025-10-26T18:06:29.557202Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.573050Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:29.594286Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 82 partNo 4 count 4 size 7988760
2025-10-26T18:06:29.599999Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.600048Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.600586Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 89 partNo : 0 messageNo: 177 size: 511959
2025-10-26T18:06:29.601076Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 89 partNo : 1 messageNo: 177 size: 511959
2025-10-26T18:06:29.601572Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 89 partNo : 2 messageNo: 177 size: 511959
2025-10-26T18:06:29.607406Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:29.608300Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 89 partNo : 3 messageNo: 177 size: 511959
2025-10-26T18:06:29.608807Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 89 partNo : 4 messageNo: 177 size: 205040
2025-10-26T18:06:29.608856Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 89 partNo : 4 messageNo: 177 size 205040 offset: -1
2025-10-26T18:06:29.611421Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.611460Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.612375Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 178  requestId: <none> cookie: 90
2025-10-26T18:06:29.614093Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.634457Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.634507Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.637430Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:29.639158Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 90 partNo : 0 messageNo: 179 size: 511959
2025-10-26T18:06:29.639685Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 90 partNo : 1 messageNo: 179 size: 511959
2025-10-26T18:06:29.640215Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 90 partNo : 2 messageNo: 179 size: 511959
2025-10-26T18:06:29.640651Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 90 partNo : 3 messageNo: 179 size: 511959
2025-10-26T18:06:29.640853Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 90 partNo : 4 messageNo: 179 size: 205040
2025-10-26T18:06:29.640868Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 90 partNo : 4 messageNo: 179 size 205040 offset: -1
2025-10-26T18:06:29.656709Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:29.658300Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 82 count 4 size 7988760 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:29.658481Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 21 size 7988782
2025-10-26T18:06:29.658518Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 6758733 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:29.658529Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 82 size 7988760
2025-10-26T18:06:29.658555Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.658611Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 86, partNo: 0, Offset: 85 is stored on disk
2025-10-26T18:06:29.658632Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.658672Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 86, partNo: 1, Offset: 85 is stored on disk
2025-10-26T18:06:29.658693Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.658728Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 86, partNo: 2, Offset: 85 is stored on disk
2025-10-26T18:06:29.658746Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.658782Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 86, partNo: 3, Offset: 85 is stored on disk
2025-10-26T18:06:29.658814Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.658852Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 86, partNo: 4, Offset: 85 is stored on disk
2025-10-26T18:06:29.658871Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.658912Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 87, partNo: 0, Offset: 86 is stored on disk
2025-10-26T18:06:29.658928Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.658973Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 87, partNo: 1, Offset: 86 is stored on disk
2025-10-26T18:06:29.658990Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.659027Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 87, partNo: 2, Offset: 86 is stored on disk
2025-10-26T18:06:29.659047Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.659084Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 87, partNo: 3, Offset: 86 is stored on disk
2025-10-26T18:06:29.659109Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.659146Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 87, partNo: 4, Offset: 86 is stored on disk
2025-10-26T18:06:29.659162Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.659198Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 88, partNo: 0, Offset: 87 is stored on disk
2025-10-26T18:06:29.659214Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.659247Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 88, partNo: 1, Offset: 87 is stored on disk
2025-10-26T18:06:29.659261Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.659298Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 88, partNo: 2, Offset: 87 is stored on disk
2025-10-26T18:06:29.659315Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.659353Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 88, partNo: 3, Offset: 87 is stored on disk
2025-10-26T18:06:29.659375Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.659412Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 88, partNo: 4, Offset: 87 is stored on disk
2025-10-26T18:06:29.660555Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.660614Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.660806Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 171  requestId: <none> cookie: 86
2025-10-26T18:06:29.660966Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 173  requestId: <none> cookie: 87
2025-10-26T18:06:29.661045Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 175  requestId: <none> cookie: 88
2025-10-26T18:06:29.661155Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 21 size 7988782 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:29.661179Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:29.661278Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.661365Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.664523Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.669903Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 86 offsets: 85 already_written: false write_statistics { persist_duration_ms: 244 queued_in_partition_duration_ms: 79 }
2025-10-26T18:06:29.669974Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 86
2025-10-26T18:06:29.670025Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:29.678083Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 87 offsets: 86 already_written: false write_statistics { persist_duration_ms: 196 queued_in_partition_duration_ms: 111 }
2025-10-26T18:06:29.678137Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 87
2025-10-26T18:06:29.681374Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 88 offsets: 87 already_written: false write_statistics { persist_duration_ms: 133 queued_in_partition_duration_ms: 151 }
2025-10-26T18:06:29.681423Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 88
2025-10-26T18:06:29.685168Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 96 to 96
2025-10-26T18:06:29.690465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:29.690515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 96
2025-10-26T18:06:29.691218Z :DEBUG: [/Root] Take Data. Partition 0. Read: {3, 0} (24-24)
2025-10-26T18:06:29.691277Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] The application data is transferred to the client. Number of messages 1, size 2252802 bytes
2025-10-26T18:06:29.691314Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] In ContinueReadingDataImpl, ReadSizeBudget = 9011865, ReadSizeServerDelta = -8487577
2025-10-26T18:06:29.691622Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-26T18:06:29.691666Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252802 bytes.. Information: { Offset: 24 ProducerId: "src_id" SeqNo: 25 CreateTime: 2025-10-26T18:06:21.417000Z WriteTime: 2025-10-26T18:06:21.906000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:50340", "server": "ipv6:[::1]:50340" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-26T18:06:29.690251Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 89 partNo 0
2025-10-26T18:06:29.690310Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 89 partNo 1
2025-10-26T18:06:29.690330Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 89 partNo 2
2025-10-26T18:06:29.690349Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 89 partNo 3
2025-10-26T18:06:29.690370Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 89 partNo 4
2025-10-26T18:06:29.694128Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 1, data# { read_request { bytes_size: 9011865 } }
2025-10-26T18:06:29.694315Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 got read request: guid# 745b05c-4d764e5-a2fe84a2-fa2f9aa1
2025-10-26T18:06:29.694371Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 performing read request: guid# ebb781ff-5c203f0e-4082428a-ba75da6a, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 57, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-26T18:06:29.694443Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 57 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 25 EndOffset 73 ClientCommitOffset 0 committedOffset 0 Guid ebb781ff-5c203f0e-4082428a-ba75da6a
2025-10-26T18:06:29.695327Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.695373Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.695446Z node 2 :PERSQUEUE DEBUG: Read proxy: bootstrap for direct read id: 0
2025-10-26T18:06:29.710394Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 97 to 97
2025-10-26T18:06:29.712708Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:29.712751Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 97
2025-10-26T18:06:29.713147Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:29.715285Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 89 partNo 4 FormedBlobsCount 0 NewHead: Offset 88 PartNo 0 PackedSize 2253230 count 1 nextOffset 89 batches 5
2025-10-26T18:06:29.715435Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 90 partNo 0
2025-10-26T18:06:29.715475Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 90 partNo 1
2025-10-26T18:06:29.715495Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 90 partNo 2
2025-10-26T18:06:29.715513Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 90 partNo 3
2025-10-26T18:06:29.715530Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 90 partNo 4
2025-10-26T18:06:29.724603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 98 to 98
2025-10-26T18:06:29.730230Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:29.730279Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 98
2025-10-26T18:06:29.730310Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:29.730667Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:29.739914Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 90 partNo 4 FormedBlobsCount 0 NewHead: Offset 88 PartNo 0 PackedSize 4506428 count 2 nextOffset 90 batches 9
2025-10-26T18:06:29.740700Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 88,2 HeadOffset 86 endOffset 88 curOffset 90 d0000000000_00000000000000000088_00000_0000000002_00008| size 4506418 WTime 1761501989740
2025-10-26T18:06:29.747878Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:29.749791Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 180  requestId: <none> cookie: 91
2025-10-26T18:06:29.750636Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.749955Z node 2 :PERSQUEUE DEBUG: read cookie 11 Topic 'rt3.dc1--test-topic' partition 0 user user offset 25 count 57 size 524288 endOffset 88 max time lag 0ms effective offset 25
2025-10-26T18:06:29.750220Z node 2 :PERSQUEUE DEBUG: read cookie 11 added 2 blobs, size 8295604 count 3 last offset 28
2025-10-26T18:06:29.750295Z node 2 :PERSQUEUE DEBUG: Reading cookie 11. Send blob request.
2025-10-26T18:06:29.763452Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.763496Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.764052Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 91 partNo : 0 messageNo: 181 size: 511959
2025-10-26T18:06:29.764546Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 91 partNo : 1 messageNo: 181 size: 511959
2025-10-26T18:06:29.765020Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 91 partNo : 2 messageNo: 181 size: 511959
2025-10-26T18:06:29.765717Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 91 partNo : 3 messageNo: 181 size: 511959
2025-10-26T18:06:29.772334Z node 2 :PERSQUEUE DEBUG: No blob in L1. Partition 0 offset 21 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:29.772366Z node 2 :PERSQUEUE DEBUG: No blob in L1. Partition 0 offset 25 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:29.772404Z node 2 :PERSQUEUE DEBUG: Reading cookie 11. Have to read 2 of 2 from KV
2025-10-26T18:06:29.772936Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 91 partNo : 4 messageNo: 181 size: 205040
2025-10-26T18:06:29.772960Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 91 partNo : 4 messageNo: 181 size 205040 offset: -1
2025-10-26T18:06:29.775377Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21
2025-10-26T18:06:29.775407Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 25
2025-10-26T18:06:29.776583Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:29.782149Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505822 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:29.782212Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.782282Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 89, partNo: 0, Offset: 88 is stored on disk
2025-10-26T18:06:29.782305Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.782352Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 89, partNo: 1, Offset: 88 is stored on disk
2025-10-26T18:06:29.782369Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.782399Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 89, partNo: 2, Offset: 88 is stored on disk
2025-10-26T18:06:29.782409Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.782435Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 89, partNo: 3, Offset: 88 is stored on disk
2025-10-26T18:06:29.782472Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.782512Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 89, partNo: 4, Offset: 88 is stored on disk
2025-10-26T18:06:29.782527Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.782563Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 90, partNo: 0, Offset: 89 is stored on disk
2025-10-26T18:06:29.782580Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.782628Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 90, partNo: 1, Offset: 89 is stored on disk
2025-10-26T18:06:29.782643Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.782684Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 90, partNo: 2, Offset: 89 is stored on disk
2025-10-26T18:06:29.782701Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.782738Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 90, partNo: 3, Offset: 89 is stored on disk
2025-10-26T18:06:29.782774Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:29.782812Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 90, partNo: 4, Offset: 89 is stored on disk
2025-10-26T18:06:29.786039Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 177  requestId: <none> cookie: 89
2025-10-26T18:06:29.786151Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 179  requestId: <none> cookie: 90
2025-10-26T18:06:29.787501Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:29.787581Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.787716Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.788015Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.788071Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.789334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 89 offsets: 88 already_written: false write_statistics { persist_duration_ms: 91 queued_in_partition_duration_ms: 82 }
2025-10-26T18:06:29.789374Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 89
2025-10-26T18:06:29.789400Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:29.806236Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 99 to 99
2025-10-26T18:06:29.808192Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 91 partNo 0
2025-10-26T18:06:29.808501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:29.808544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 99
2025-10-26T18:06:29.808576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:29.809736Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:29.809840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 90 offsets: 89 already_written: false write_statistics { persist_duration_ms: 68 queued_in_partition_duration_ms: 73 }
2025-10-26T18:06:29.809874Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 90
2025-10-26T18:06:29.809898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:29.830883Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 91 partNo 0 result is x0000000000_00000000000000000086_00002_0000000004_00014 size 7988778
2025-10-26T18:06:29.832575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write 1 messages with Id from 100 to 100
2025-10-26T18:06:29.834005Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 91 partNo 1
2025-10-26T18:06:29.834070Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 91 partNo 2
2025-10-26T18:06:29.834090Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 91 partNo 3
2025-10-26T18:06:29.834111Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 91 partNo 4
2025-10-26T18:06:29.834183Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000086_00002_0000000004_00014 new key d0000000000_00000000000000000086_00002_0000000004_00014 size 7988778 WTime 1761501989833
2025-10-26T18:06:29.836096Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: try to update token
2025-10-26T18:06:29.836137Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Send 1 message(s) (0 left), first sequence number is 100
2025-10-26T18:06:29.836170Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-26T18:06:29.837213Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:4803" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:4803. " }
2025-10-26T18:06:29.837335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: close. Timeout = 18446744073709551 ms
2025-10-26T18:06:29.885628Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 91 partNo 4 FormedBlobsCount 1 NewHead: Offset 90 PartNo 0 PackedSize 2253230 count 1 nextOffset 91 batches 5
2025-10-26T18:06:29.886597Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 90,1 HeadOffset 86 endOffset 90 curOffset 91 d0000000000_00000000000000000090_00000_0000000001_00004| size 2253220 WTime 1761501989886
2025-10-26T18:06:29.890775Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:29.910751Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 86 partNo 2 count 4 size 7988778
2025-10-26T18:06:29.926092Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 182  requestId: <none> cookie: 92
2025-10-26T18:06:29.927256Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:29.959131Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:29.961563Z node 2 :PERSQUEUE DEBUG: Got results. 2 of 2 from KV. Status 1
2025-10-26T18:06:29.961604Z node 2 :PERSQUEUE DEBUG: Got results. result 0 from KV. Status 0
2025-10-26T18:06:29.961614Z node 2 :PERSQUEUE DEBUG: Got results. result 1 from KV. Status 0
2025-10-26T18:06:29.961654Z node 2 :PERSQUEUE DEBUG: Prefetched blob in L1. Partition 0 offset 21 count 4 size 7988782 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:29.961672Z node 2 :PERSQUEUE DEBUG: Prefetched blob in L1. Partition 0 offset 25 count 3 size 8295604 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:29.964210Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 72 size 8295660
2025-10-26T18:06:29.964240Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 size 7988782
2025-10-26T18:06:29.964258Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 75 size 7988804
2025-10-26T18:06:29.964269Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 size 8295604
2025-10-26T18:06:29.967466Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.967550Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.972300Z node 2 :PERSQUEUE DEBUG: FormAnswer 2
2025-10-26T18:06:29.974025Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 92 partNo : 0 messageNo: 183 size: 511959
2025-10-26T18:06:29.974678Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 92 partNo : 1 messageNo: 183 size: 511959
2025-10-26T18:06:29.975181Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 92 partNo : 2 messageNo: 183 size: 511959
2025-10-26T18:06:29.976136Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 92 partNo : 3 messageNo: 183 size: 511959
2025-10-26T18:06:29.976398Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 92 partNo : 4 messageNo: 183 size: 205040
2025-10-26T18:06:29.976428Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 92 partNo : 4 messageNo: 183 size 205040 offset: -1
2025-10-26T18:06:29.977862Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:29.978750Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:29.978781Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:29.979391Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 72 size 8295660 cause it's been evicted from L2. Actual L1 size: 5
2025-10-26T18:06:29.979418Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 75 size 7988804 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:29.979434Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:29.993625Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 86 count 4 size 7988778 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:29.993768Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 79 size 8295604
2025-10-26T18:06:29.993788Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 86 size 7988778
2025-10-26T18:06:29.994619Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 79 size 8295604 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:29.994654Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:30.012783Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 25 totakecount 4 count 0 size 512009 from pos 0 cbcount 1
2025-10-26T18:06:30.014949Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 25 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:30.016493Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 25 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:30.018037Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 25 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:30.019859Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 25 totakecount 3 count 1 size 717095 from pos 0 cbcount 2
2025-10-26T18:06:30.021579Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 26 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:30.023125Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 26 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:30.024695Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 26 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:30.025640Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 26 totakecount 3 count 1 size 205086 from pos 0 cbcount 1
2025-10-26T18:06:30.026992Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 27 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-26T18:06:30.028571Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 27 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:30.030245Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 27 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:30.031833Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 27 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-26T18:06:30.032785Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 27 totakecount 3 count 1 size 205086 from pos 0 cbcount 1
2025-10-26T18:06:30.037821Z node 2 :PERSQUEUE DEBUG: FormAnswer processing batch offset 28 totakecount 3 count 0 size 2047975 from pos 0 cbcount 4
2025-10-26T18:06:30.041204Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252911 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:30.041239Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.041277Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 91, partNo: 0, Offset: 90 is stored on disk
2025-10-26T18:06:30.041288Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.041311Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 91, partNo: 1, Offset: 90 is stored on disk
2025-10-26T18:06:30.041320Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.041342Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 91, partNo: 2, Offset: 90 is stored on disk
2025-10-26T18:06:30.041353Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.041377Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 91, partNo: 3, Offset: 90 is stored on disk
2025-10-26T18:06:30.041403Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.041423Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 91, partNo: 4, Offset: 90 is stored on disk
2025-10-26T18:06:30.056991Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 25
2025-10-26T18:06:30.057428Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 184  requestId: <none> cookie: 93
2025-10-26T18:06:30.057852Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 181  requestId: <none> cookie: 91
2025-10-26T18:06:30.061812Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.061931Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.064967Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 92 partNo 0
2025-10-26T18:06:30.065030Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 92 partNo 1
2025-10-26T18:06:30.065049Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 92 partNo 2
2025-10-26T18:06:30.065070Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 92 partNo 3
2025-10-26T18:06:30.065087Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 92 partNo 4
2025-10-26T18:06:30.074224Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 91 offsets: 90 already_written: false write_statistics { persist_duration_ms: 206 queued_in_partition_duration_ms: 58 }
2025-10-26T18:06:30.074318Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 91
2025-10-26T18:06:30.074368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Estimated memory usage got back to normal 20275218[B]
2025-10-26T18:06:30.102177Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:30.119792Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:30.137272Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.137313Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.138066Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 93 partNo : 0 messageNo: 185 size: 511959
2025-10-26T18:06:30.146583Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 93 partNo : 1 messageNo: 185 size: 511959
2025-10-26T18:06:30.147250Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 93 partNo : 2 messageNo: 185 size: 511959
2025-10-26T18:06:30.148206Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 93 partNo : 3 messageNo: 185 size: 511959
2025-10-26T18:06:30.148863Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 93 partNo : 4 messageNo: 185 size: 205040
2025-10-26T18:06:30.148922Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 93 partNo : 4 messageNo: 185 size 205040 offset: -1
2025-10-26T18:06:30.151220Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.151252Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.166400Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 92 partNo 4 FormedBlobsCount 0 NewHead: Offset 91 PartNo 0 PackedSize 2253230 count 1 nextOffset 92 batches 5
2025-10-26T18:06:30.167449Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 91,1 HeadOffset 90 endOffset 91 curOffset 92 d0000000000_00000000000000000091_00000_0000000001_00004| size 2253220 WTime 1761501990166
2025-10-26T18:06:30.178092Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 90 Result { Offset: 25 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 26 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501981932 CreateTimestampMS: 1761501981451 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 26 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 27 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501982253 CreateTimestampMS: 1761501981485 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } Result { Offset: 27 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 28 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1761501982594 CreateTimestampMS: 1761501982276 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 2 BlobsFromCache: 0 SizeLag: 145127388 RealReadOffset: 27 WaitQuotaTimeMs: 0 } Cookie: 25 }
2025-10-26T18:06:30.185306Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:30.185774Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 186  requestId: <none> cookie: 94
2025-10-26T18:06:30.189185Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.204095Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 28 endOffset 90
2025-10-26T18:06:30.204178Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 90 ReadOffset 28 ReadGuid ebb781ff-5c203f0e-4082428a-ba75da6a has messages 1
2025-10-26T18:06:30.209820Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 28, endOffset# 90, WTime# 1761501982594, sizeLag# 145127388
2025-10-26T18:06:30.209863Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1TEvPartitionReady. Aval parts: 0
2025-10-26T18:06:30.209947Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 read done: guid# ebb781ff-5c203f0e-4082428a-ba75da6a, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 6758904
2025-10-26T18:06:30.209991Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 response to read: guid# ebb781ff-5c203f0e-4082428a-ba75da6a
2025-10-26T18:06:30.223137Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:30.229015Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.229101Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.229800Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:30.230115Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 94 partNo : 0 messageNo: 187 size: 511959
2025-10-26T18:06:30.232634Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 Process answer. Aval parts: 1
2025-10-26T18:06:30.233362Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 94 partNo : 1 messageNo: 187 size: 511959
2025-10-26T18:06:30.233990Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 94 partNo : 2 messageNo: 187 size: 511959
2025-10-26T18:06:30.234502Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 94 partNo : 3 messageNo: 187 size: 511959
2025-10-26T18:06:30.235339Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 94 partNo : 4 messageNo: 187 size: 205040
2025-10-26T18:06:30.235391Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 94 partNo : 4 messageNo: 187 size 205040 offset: -1
2025-10-26T18:06:30.237202Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.237233Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.237828Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 188  requestId: <none> cookie: 95
2025-10-26T18:06:30.239487Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.249179Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:30.257638Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:30.264015Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.264063Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.266600Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 95 partNo : 0 messageNo: 189 size: 511959
2025-10-26T18:06:30.267683Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 95 partNo : 1 messageNo: 189 size: 511959
2025-10-26T18:06:30.268565Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 95 partNo : 2 messageNo: 189 size: 511959
2025-10-26T18:06:30.270692Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Got ReadResponse, serverBytesSize = 6758904, now ReadSizeBudget = 0, ReadSizeServerDelta = -6234616
2025-10-26T18:06:30.271038Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (25-25)
2025-10-26T18:06:30.271117Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Returning serverBytesSize = 6758904 to budget
2025-10-26T18:06:30.269433Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 95 partNo : 3 messageNo: 189 size: 511959
2025-10-26T18:06:30.269984Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 95 partNo : 4 messageNo: 189 size: 205040
2025-10-26T18:06:30.270006Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 95 partNo : 4 messageNo: 189 size 205040 offset: -1
2025-10-26T18:06:30.271829Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252911 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:30.271865Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.271915Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 92, partNo: 0, Offset: 91 is stored on disk
2025-10-26T18:06:30.271936Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.271996Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 92, partNo: 1, Offset: 91 is stored on disk
2025-10-26T18:06:30.272014Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.272059Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 92, partNo: 2, Offset: 91 is stored on disk
2025-10-26T18:06:30.272078Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.272118Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 92, partNo: 3, Offset: 91 is stored on disk
2025-10-26T18:06:30.272155Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.272194Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 92, partNo: 4, Offset: 91 is stored on disk
2025-10-26T18:06:30.272468Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.272491Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.272576Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 183  requestId: <none> cookie: 92
2025-10-26T18:06:30.274050Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.282010Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 92 offsets: 91 already_written: false write_statistics { persist_duration_ms: 206 queued_in_partition_duration_ms: 24 }
2025-10-26T18:06:30.282058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 92
2025-10-26T18:06:30.284566Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 93 partNo 0
2025-10-26T18:06:30.284626Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 93 partNo 1
2025-10-26T18:06:30.284644Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 93 partNo 2
2025-10-26T18:06:30.284659Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 93 partNo 3
2025-10-26T18:06:30.284679Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 93 partNo 4
2025-10-26T18:06:30.305911Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 93 partNo 4 FormedBlobsCount 0 NewHead: Offset 92 PartNo 0 PackedSize 2253230 count 1 nextOffset 93 batches 5
2025-10-26T18:06:30.306035Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 94 partNo 0
2025-10-26T18:06:30.306069Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 94 partNo 1
2025-10-26T18:06:30.306086Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 94 partNo 2
2025-10-26T18:06:30.306103Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 94 partNo 3
2025-10-26T18:06:30.354532Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 94 partNo 3 result is x0000000000_00000000000000000090_00000_0000000003_00015 size 8295682
2025-10-26T18:06:30.357362Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 94 partNo 4
2025-10-26T18:06:30.357633Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000090_00000_0000000003_00015 new key d0000000000_00000000000000000090_00000_0000000003_00015 size 8295682 WTime 1761501990357
2025-10-26T18:06:30.362046Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 94 partNo 4 FormedBlobsCount 1 NewHead: Offset 93 PartNo 3 PackedSize 717145 count 1 nextOffset 94 batches 2
2025-10-26T18:06:30.362144Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 95 partNo 0
2025-10-26T18:06:30.362175Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 95 partNo 1
2025-10-26T18:06:30.362193Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 95 partNo 2
2025-10-26T18:06:30.362210Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 95 partNo 3
2025-10-26T18:06:30.362227Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 95 partNo 4
2025-10-26T18:06:30.373590Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 95 partNo 4 FormedBlobsCount 0 NewHead: Offset 93 PartNo 3 PackedSize 2970343 count 2 nextOffset 95 batches 6
2025-10-26T18:06:30.374358Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 93,2 HeadOffset 90 endOffset 92 curOffset 95 d0000000000_00000000000000000093_00003_0000000002_00005| size 2970333 WTime 1761501990374
2025-10-26T18:06:30.386969Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:30.387084Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 190  requestId: <none> cookie: 96
2025-10-26T18:06:30.387461Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.398872Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.398930Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.399495Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 96 partNo : 0 messageNo: 191 size: 511959
2025-10-26T18:06:30.400054Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 96 partNo : 1 messageNo: 191 size: 511959
2025-10-26T18:06:30.400635Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 96 partNo : 2 messageNo: 191 size: 511959
2025-10-26T18:06:30.403996Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 96 partNo : 3 messageNo: 191 size: 511959
2025-10-26T18:06:30.406656Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:30.412050Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 96 partNo : 4 messageNo: 191 size: 205040
2025-10-26T18:06:30.412085Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 96 partNo : 4 messageNo: 191 size 205040 offset: -1
2025-10-26T18:06:30.421162Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:30.421977Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.422018Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.422112Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 192  requestId: <none> cookie: 97
2025-10-26T18:06:30.422457Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.426547Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 90 partNo 0 count 3 size 8295682
2025-10-26T18:06:30.439741Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.439785Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.439982Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:30.455908Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 97 partNo : 0 messageNo: 193 size: 511959
2025-10-26T18:06:30.456459Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 97 partNo : 1 messageNo: 193 size: 511959
2025-10-26T18:06:30.456993Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 97 partNo : 2 messageNo: 193 size: 511959
2025-10-26T18:06:30.457524Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 97 partNo : 3 messageNo: 193 size: 511959
2025-10-26T18:06:30.457794Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 97 partNo : 4 messageNo: 193 size: 205040
2025-10-26T18:06:30.457815Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 97 partNo : 4 messageNo: 193 size 205040 offset: -1
2025-10-26T18:06:30.482102Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:30.482628Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.482661Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.482741Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 194  requestId: <none> cookie: 98
2025-10-26T18:06:30.482950Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.505672Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.505737Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.506328Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 98 partNo : 0 messageNo: 195 size: 511959
2025-10-26T18:06:30.506832Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 98 partNo : 1 messageNo: 195 size: 511959
2025-10-26T18:06:30.507360Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 98 partNo : 2 messageNo: 195 size: 511959
2025-10-26T18:06:30.507917Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 98 partNo : 3 messageNo: 195 size: 511959
2025-10-26T18:06:30.510902Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 98 partNo : 4 messageNo: 195 size: 205040
2025-10-26T18:06:30.510937Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 98 partNo : 4 messageNo: 195 size 205040 offset: -1
2025-10-26T18:06:30.511775Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:30.517320Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:30.532813Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.532859Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.533486Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 196  requestId: <none> cookie: 99
2025-10-26T18:06:30.536782Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.552690Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-26T18:06:30.554271Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.554316Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.554921Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 99 partNo : 0 messageNo: 197 size: 511959
2025-10-26T18:06:30.555828Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 99 partNo : 1 messageNo: 197 size: 511959
2025-10-26T18:06:30.556415Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 99 partNo : 2 messageNo: 197 size: 511959
2025-10-26T18:06:30.556995Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 99 partNo : 3 messageNo: 197 size: 511959
2025-10-26T18:06:30.557251Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 99 partNo : 4 messageNo: 197 size: 205040
2025-10-26T18:06:30.557271Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 99 partNo : 4 messageNo: 197 size 205040 offset: -1
2025-10-26T18:06:30.563267Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 90 count 3 size 8295682 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:30.563458Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 6758733 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:30.563477Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563515Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 93, partNo: 0, Offset: 92 is stored on disk
2025-10-26T18:06:30.563523Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563533Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 82 size 7988760
2025-10-26T18:06:30.563542Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 93, partNo: 1, Offset: 92 is stored on disk
2025-10-26T18:06:30.563551Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563564Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 90 size 8295682
2025-10-26T18:06:30.563571Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 93, partNo: 2, Offset: 92 is stored on disk
2025-10-26T18:06:30.563578Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563597Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 93, partNo: 3, Offset: 92 is stored on disk
2025-10-26T18:06:30.563619Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563640Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 93, partNo: 4, Offset: 92 is stored on disk
2025-10-26T18:06:30.563649Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563667Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 94, partNo: 0, Offset: 93 is stored on disk
2025-10-26T18:06:30.563676Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563696Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 94, partNo: 1, Offset: 93 is stored on disk
2025-10-26T18:06:30.563703Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563721Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 94, partNo: 2, Offset: 93 is stored on disk
2025-10-26T18:06:30.563729Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563747Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 94, partNo: 3, Offset: 93 is stored on disk
2025-10-26T18:06:30.563757Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563777Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 94, partNo: 4, Offset: 93 is stored on disk
2025-10-26T18:06:30.563785Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563802Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 95, partNo: 0, Offset: 94 is stored on disk
2025-10-26T18:06:30.563811Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563830Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 95, partNo: 1, Offset: 94 is stored on disk
2025-10-26T18:06:30.563838Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563855Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 95, partNo: 2, Offset: 94 is stored on disk
2025-10-26T18:06:30.563862Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563880Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 95, partNo: 3, Offset: 94 is stored on disk
2025-10-26T18:06:30.563891Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.563910Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 95, partNo: 4, Offset: 94 is stored on disk
2025-10-26T18:06:30.569602Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-26T18:06:30.572864Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 96 partNo 0
2025-10-26T18:06:30.572919Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 96 partNo 1
2025-10-26T18:06:30.572939Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 96 partNo 2
2025-10-26T18:06:30.572957Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 96 partNo 3
2025-10-26T18:06:30.572974Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 96 partNo 4
2025-10-26T18:06:30.588427Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 96 partNo 4 FormedBlobsCount 0 NewHead: Offset 95 PartNo 0 PackedSize 2253230 count 1 nextOffset 96 batches 5
2025-10-26T18:06:30.588568Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 97 partNo 0
2025-10-26T18:06:30.588602Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 97 partNo 1
2025-10-26T18:06:30.588626Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 97 partNo 2
2025-10-26T18:06:30.588642Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 97 partNo 3
2025-10-26T18:06:30.588662Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 97 partNo 4
2025-10-26T18:06:30.601183Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 97 partNo 4 FormedBlobsCount 0 NewHead: Offset 95 PartNo 0 PackedSize 4506428 count 2 nextOffset 97 batches 9
2025-10-26T18:06:30.601299Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 98 partNo 0
2025-10-26T18:06:30.601333Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 98 partNo 1
2025-10-26T18:06:30.654964Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 98 partNo 1 result is x0000000000_00000000000000000093_00003_0000000004_00014 size 7988782
2025-10-26T18:06:30.658040Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 98 partNo 2
2025-10-26T18:06:30.658105Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 98 partNo 3
2025-10-26T18:06:30.658123Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 98 partNo 4
2025-10-26T18:06:30.658188Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000093_00003_0000000004_00014 new key d0000000000_00000000000000000093_00003_0000000004_00014 size 7988782 WTime 1761501990657
2025-10-26T18:06:30.683751Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 98 partNo 4 FormedBlobsCount 1 NewHead: Offset 97 PartNo 1 PackedSize 1741199 count 1 nextOffset 98 batches 4
2025-10-26T18:06:30.683881Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 99 partNo 0
2025-10-26T18:06:30.683912Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 99 partNo 1
2025-10-26T18:06:30.683930Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 99 partNo 2
2025-10-26T18:06:30.683946Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 99 partNo 3
2025-10-26T18:06:30.683966Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 99 partNo 4
2025-10-26T18:06:30.717716Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 99 partNo 4 FormedBlobsCount 0 NewHead: Offset 97 PartNo 1 PackedSize 3994397 count 2 nextOffset 99 batches 8
2025-10-26T18:06:30.718731Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 97,2 HeadOffset 93 endOffset 95 curOffset 99 d0000000000_00000000000000000097_00001_0000000002_00007| size 3994387 WTime 1761501990718
2025-10-26T18:06:30.728017Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.728215Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.729590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 93 offsets: 92 already_written: false write_statistics { persist_duration_ms: 275 queued_in_partition_duration_ms: 100 }
2025-10-26T18:06:30.729656Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 93
2025-10-26T18:06:30.727249Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 185  requestId: <none> cookie: 93
2025-10-26T18:06:30.727359Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 187  requestId: <none> cookie: 94
2025-10-26T18:06:30.727417Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 189  requestId: <none> cookie: 95
2025-10-26T18:06:30.727487Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.727511Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.727570Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 82 size 7988760 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:30.727585Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:30.731339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 94 offsets: 93 already_written: false write_statistics { persist_duration_ms: 253 queued_in_partition_duration_ms: 68 }
2025-10-26T18:06:30.731370Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 94
2025-10-26T18:06:30.731126Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:30.733492Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.734487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 95 offsets: 94 already_written: false write_statistics { persist_duration_ms: 198 queued_in_partition_duration_ms: 91 }
2025-10-26T18:06:30.734561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 95
2025-10-26T18:06:30.756261Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough blob. Partition 0 offset 93 partNo 3 count 4 size 7988782
2025-10-26T18:06:30.768385Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.767791Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 198  requestId: <none> cookie: 100
2025-10-26T18:06:30.779180Z node 2 :PERSQUEUE DEBUG: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-26T18:06:30.779251Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-26T18:06:30.780001Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 100 partNo : 0 messageNo: 199 size: 511959
2025-10-26T18:06:30.780701Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 100 partNo : 1 messageNo: 199 size: 511959
2025-10-26T18:06:30.781233Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 100 partNo : 2 messageNo: 199 size: 511959
2025-10-26T18:06:30.781783Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 100 partNo : 3 messageNo: 199 size: 511959
2025-10-26T18:06:30.782030Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 100 partNo : 4 messageNo: 199 size: 205040
2025-10-26T18:06:30.782054Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 100 partNo : 4 messageNo: 199 size 205040 offset: -1
2025-10-26T18:06:30.806777Z node 2 :PERSQUEUE DEBUG: Caching head blob in L1. Partition 0 offset 93 count 4 size 7988782 actorID [2:7565593381887295230:4300]
2025-10-26T18:06:30.806931Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 9011644 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:30.806956Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.806999Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 96, partNo: 0, Offset: 95 is stored on disk
2025-10-26T18:06:30.807012Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807032Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 96, partNo: 1, Offset: 95 is stored on disk
2025-10-26T18:06:30.807039Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807060Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 96, partNo: 2, Offset: 95 is stored on disk
2025-10-26T18:06:30.807071Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807090Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 96, partNo: 3, Offset: 95 is stored on disk
2025-10-26T18:06:30.807115Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807135Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 96, partNo: 4, Offset: 95 is stored on disk
2025-10-26T18:06:30.807144Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807163Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 97, partNo: 0, Offset: 96 is stored on disk
2025-10-26T18:06:30.807171Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807191Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 97, partNo: 1, Offset: 96 is stored on disk
2025-10-26T18:06:30.807200Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807220Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 97, partNo: 2, Offset: 96 is stored on disk
2025-10-26T18:06:30.807228Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807248Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 97, partNo: 3, Offset: 96 is stored on disk
2025-10-26T18:06:30.807259Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807278Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 97, partNo: 4, Offset: 96 is stored on disk
2025-10-26T18:06:30.807286Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807307Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 98, partNo: 0, Offset: 97 is stored on disk
2025-10-26T18:06:30.807316Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807336Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 98, partNo: 1, Offset: 97 is stored on disk
2025-10-26T18:06:30.807343Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807377Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 98, partNo: 2, Offset: 97 is stored on disk
2025-10-26T18:06:30.807389Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807407Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 98, partNo: 3, Offset: 97 is stored on disk
2025-10-26T18:06:30.807418Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807438Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 98, partNo: 4, Offset: 97 is stored on disk
2025-10-26T18:06:30.807446Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807465Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 99, partNo: 0, Offset: 98 is stored on disk
2025-10-26T18:06:30.807471Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807489Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 99, partNo: 1, Offset: 98 is stored on disk
2025-10-26T18:06:30.807496Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807515Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 99, partNo: 2, Offset: 98 is stored on disk
2025-10-26T18:06:30.807521Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807537Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 99, partNo: 3, Offset: 98 is stored on disk
2025-10-26T18:06:30.807547Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.807567Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 99, partNo: 4, Offset: 98 is stored on disk
2025-10-26T18:06:30.809829Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 21 size 7988782
2025-10-26T18:06:30.809850Z node 2 :PERSQUEUE DEBUG: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 93 size 7988782
2025-10-26T18:06:30.812325Z node 2 :PERSQUEUE DEBUG: Erasing blob in L1. Partition 0 offset 21 size 7988782 cause it's been evicted from L2. Actual L1 size: 4
2025-10-26T18:06:30.812362Z node 2 :PERSQUEUE NOTICE: Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-26T18:06:30.816739Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 100 partNo 0
2025-10-26T18:06:30.816807Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 100 partNo 1
2025-10-26T18:06:30.816827Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 100 partNo 2
2025-10-26T18:06:30.816849Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 100 partNo 3
2025-10-26T18:06:30.816865Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 100 partNo 4
2025-10-26T18:06:30.819404Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 191  requestId: <none> cookie: 96
2025-10-26T18:06:30.819505Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 193  requestId: <none> cookie: 97
2025-10-26T18:06:30.819563Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 195  requestId: <none> cookie: 98
2025-10-26T18:06:30.819619Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 197  requestId: <none> cookie: 99
2025-10-26T18:06:30.821837Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.821958Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.822001Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.822039Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.826091Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 96 offsets: 95 already_written: false write_statistics { persist_duration_ms: 234 queued_in_partition_duration_ms: 157 }
2025-10-26T18:06:30.826138Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 96
2025-10-26T18:06:30.827726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 97 offsets: 96 already_written: false write_statistics { persist_duration_ms: 217 queued_in_partition_duration_ms: 129 }
2025-10-26T18:06:30.827760Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 97
2025-10-26T18:06:30.828928Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 98 offsets: 97 already_written: false write_statistics { persist_duration_ms: 204 queued_in_partition_duration_ms: 90 }
2025-10-26T18:06:30.828970Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 98
2025-10-26T18:06:30.832118Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 99 offsets: 98 already_written: false write_statistics { persist_duration_ms: 121 queued_in_partition_duration_ms: 123 }
2025-10-26T18:06:30.832148Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 99
2025-10-26T18:06:30.841093Z node 2 :PERSQUEUE DEBUG: Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 100 partNo 4 FormedBlobsCount 0 NewHead: Offset 99 PartNo 0 PackedSize 2253230 count 1 nextOffset 100 batches 5
2025-10-26T18:06:30.842219Z node 2 :PERSQUEUE DEBUG: Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 99,1 HeadOffset 97 endOffset 99 curOffset 100 d0000000000_00000000000000000099_00000_0000000001_00004| size 2253220 WTime 1761501990840
2025-10-26T18:06:30.846688Z node 2 :PERSQUEUE DEBUG: CacheProxy. Passthrough write request to KV
2025-10-26T18:06:30.865851Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252911 WriteNewSizeFromSupportivePartitions# 0
2025-10-26T18:06:30.865900Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.865965Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 100, partNo: 0, Offset: 99 is stored on disk
2025-10-26T18:06:30.865979Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.866004Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 100, partNo: 1, Offset: 99 is stored on disk
2025-10-26T18:06:30.866013Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.866037Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 100, partNo: 2, Offset: 99 is stored on disk
2025-10-26T18:06:30.866046Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.866069Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 100, partNo: 3, Offset: 99 is stored on disk
2025-10-26T18:06:30.866102Z node 2 :PERSQUEUE DEBUG: TPartition::ReplyWrite. Partition: 0
2025-10-26T18:06:30.866139Z node 2 :PERSQUEUE DEBUG: Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 100, partNo: 4, Offset: 99 is stored on disk
2025-10-26T18:06:30.866161Z node 2 :PERSQUEUE DEBUG: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 199  requestId: <none> cookie: 100
2025-10-26T18:06:30.866924Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-26T18:06:30.871764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session got write response: sequence_numbers: 100 offsets: 99 already_written: false write_statistics { persist_duration_ms: 49 queued_in_partition_duration_ms: 30 }
2025-10-26T18:06:30.871819Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: acknoledged message 100
2025-10-26T18:06:30.944984Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session will now close
2025-10-26T18:06:30.945066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: aborting
2025-10-26T18:06:30.945640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: gracefully shut down, all writes complete
2025-10-26T18:06:30.945709Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0] Write session: destroy
2025-10-26T18:06:30.946008Z :INFO: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] Closing read session. Close timeout: 0.000000s
2025-10-26T18:06:30.946058Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:27:0
2025-10-26T18:06:30.946099Z :INFO: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] Counters: { Errors: 0 CurrentSessionLifetimeMs: 12618 BytesRead: 58572842 MessagesRead: 26 BytesReadCompressed: 58572842 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 4505604 BytesInflightTotal: 6758406 MessagesInflight: 3 }
2025-10-26T18:06:30.946204Z :NOTICE: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Close with zero timeout " }
2025-10-26T18:06:30.946265Z :DEBUG: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] [] Abort session to cluster
2025-10-26T18:06:30.946586Z :INFO: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Closing read session. Close timeout: 0.000000s
2025-10-26T18:06:30.946615Z :INFO: [/Invalid] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-26T18:06:30.946640Z :INFO: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Counters: { Errors: 1 CurrentSessionLifetimeMs: 12581 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-26T18:06:30.946670Z :INFO: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Closing read session. Close timeout: 0.000000s
2025-10-26T18:06:30.946699Z :INFO: [/Invalid] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-26T18:06:30.946727Z :INFO: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Counters: { Errors: 1 CurrentSessionLifetimeMs: 12581 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-26T18:06:30.946800Z :NOTICE: [/Invalid] [/Invalid] [20f4bfe2-70020106-e7fec74b-54eab5d5] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-26T18:06:30.946966Z :INFO: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] Closing read session. Close timeout: 0.000000s
2025-10-26T18:06:30.946991Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:27:0
2025-10-26T18:06:30.947020Z :INFO: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] Counters: { Errors: 0 CurrentSessionLifetimeMs: 12619 BytesRead: 58572842 MessagesRead: 26 BytesReadCompressed: 58572842 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 4505604 BytesInflightTotal: 6758406 MessagesInflight: 3 }
2025-10-26T18:06:30.947065Z :NOTICE: [/Root] [/Root] [22ddf4c8-ade65773-f5e1d156-65f30663] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-26T18:06:30.947422Z node 1 :PQ_WRITE_PROXY DEBUG: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read done: success: 0 data: 
2025-10-26T18:06:30.947458Z node 1 :PQ_WRITE_PROXY INFO: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc read failed
2025-10-26T18:06:30.947487Z node 1 :PQ_WRITE_PROXY INFO: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 grpc closed
2025-10-26T18:06:30.947506Z node 1 :PQ_WRITE_PROXY INFO: session v1 cookie: 2 sessionId: src_id|7b805ba2-4b3cbd69-5747df04-a8895da0_0 is DEAD
2025-10-26T18:06:30.948534Z node 1 :PQ_WRITE_PROXY DEBUG: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-26T18:06:30.948686Z node 1 :PQ_READ_PROXY DEBUG: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read done: success# 0, data# {  }
2025-10-26T18:06:30.948702Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc read failed
2025-10-26T18:06:30.948732Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 grpc closed
2025-10-26T18:06:30.948776Z node 1 :PQ_READ_PROXY INFO: session cookie 1 consumer shared/user session shared/user_1_1_1719487083443976093_v1 is DEAD
2025-10-26T18:06:30.950131Z node 1 :PERSQUEUE_READ_BALANCER INFO: balancer: [72075186224037893] topic rt3.dc1--test-topic pipe [1:7565593391036007795:8438] disconnected; active server actors: 1
2025-10-26T18:06:30.950187Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: balancer: [72075186224037893] topic rt3.dc1--test-topic pipe [1:7565593391036007795:8438] client user disconnected session shared/user_1_1_1719487083443976093_v1
2025-10-26T18:06:30.951125Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 server disconnected, pipe [1:7565593391036007828:8439] destroyed
2025-10-26T18:06:30.951169Z node 2 :PERSQUEUE DEBUG: [PQ: 72075186224037892, Partition: 0, State: StateIdle] TPartition::DropOwner.
2025-10-26T18:06:30.951175Z node 2 :PERSQUEUE DEBUG: PQ: Destroy direct read session shared/user_1_1_1719487083443976093_v1
2025-10-26T18:06:30.951200Z node 2 :PERSQUEUE DEBUG: Tablet 72075186224037892 server disconnected, pipe [1:7565593391036007800:8440] destroyed
2025-10-26T18:06:30.951302Z node 2 :PQ_READ_PROXY DEBUG: Direct read cache: server session deregistered: shared/user_1_1_1719487083443976093_v1
