Whisper Messaging

From Status Wiki
This is the approved revision of this page, as well as being the most recent.
Jump to: navigation, search

Initialisation

Whisper is initialised here in our core.cljs file.

Create filters for all groups.

In order to do this we first generate symmetric keys using the group id as a password (shh.generateSymKeyFromPassword). Currently this key is used only as a “topic” for messages in group chat (that’s how we distinguish group chats from each other). In whisper v2 we were allowed to use topics of any length, that’s why it was possible to pass topic option to shh.newMessageFilter. But in whisper v5 topic can be only 4 bytes, which is not enough for our purposes. When we have symmetric key we can start listening to new messages using shh.newMessageFilter with next parameters:

{
 "sym": "generated sym key",
 "topics": ["hardcoded status topic 0xaabb11ee"]
}

Create filter for 1-1 chats

The same shh.newMessageFilter is used, parameters are

{
 "asym": "user's public key",
 "topics": ["0xaabb11ee"]
}

Only one filter is created for all incoming 1-1 messages as we can distinguish senders by their signature.

Create filters for profile updates

Unlike in group messages filters, we know who will send messages and can use sig option here. Still symmetric key is used as messages are broadcasted to all contacts. So shh.newMessageFilter parameters are:

{
 "sig": "user's public key",
 "sym": "key generated from hardcoded string 'status-discovery'",
 "topics": ["0xaabb11ee"]
}

Create delivery loop (will be described in separate doc)

Send :online message

Parameters

{
;; web3 object
:web3                        web3
;; user's public key
:identity                    public-key
;; vector of groups, where group is {:group-id "id" :keypair {:private "" :public ""}}
:groups                      groups
;; function which will be called on incoming message,
;; first parameter is message type, second payload
:callback                    #(dispatch [:incoming-message %1 %2])
;; if message require ack and it wasn't received during :ack-not-received-s-interval after previous
;; attempt to send message message will be sent again. Should be bigger than default-ttl.
;; In seconds.
:ack-not-received-s-interval 125
;; default ttl for all messages in seconds
:default-ttl                              120
;; interval for sending :online message
:send-online-s-interval         180
;; custom ttl config for specific message types
:ttl-config                  { ;; we don't expect to receive acc for  :public-group-message, that's why ttl is
                                     ;;bigger  than default. This increases chances for message
                                     ;; delivery, though guaranties nothing.
                                     :public-group-message 2400}
;; Number of attempts of sending message if it requires ack.
:max-attempts-number         3
;; Interval for delivery loop. Details will be described in separate doc.
:delivery-loop-ms-interval   500
;; Keypair which is used for sending profile updates and discovery
:profile-keypair             {:public  updates-public-key
                                        :private updates-private-key}
;; messages from db that require ack but haven't received it
:pending-messages            (pending-messages/get-all)
;; all contacts which can send profile updates/discovery
:contacts                          all-contacts
;; callback for errors in `shh.post` call
:post-error-callback         #(dispatch [::post-error %])}

Delivery

  1. For each new message we create an entry of the :pending-message entity and store it in db
  2. Each new message is added to status-im.protocol.web3.delivery/messages map
  3. All :pending-message entries from db are added to status-im.protocol.web3.delivery/messages on whisper initialisation
  4. All messages from status-im.protocol.web3.delivery/messages are sent over Whisper inside delivery loop
  5. Messages which don’t require ack are removed from both :pending-message and status-im.protocol.web3.delivery/messages after successful call to shh.post
  6. Messages that require ack are removed from pending-message and status-im.protocol.web3.delivery/messages only after receiving ack.

Delivery loop

The delivery loop is started here in our delivery.cljs file. Parameters:

{;; function which sends :online message
;;https://github.com/status-im/status-react/blob/cede0957467739fb660f70944dd0fd2f2580e438/src/status_im/protocol/core.cljs#L112
:online-message    fn
;; other parameter are same as for whisper initialisation
:delivery-loop-ms-interval ...
:ack-not-received-s-interval  ...
:max-attempts-number ...
:default-ttl ...
:send-online-s-interval ...}

Delivery loop is running with delivery-loop-ms-interval interval, it is 500ms by default. In order to check if message should be sent in the current iteration of delivery loop we check message with https://github.com/status-im/status-react/blob/develop/src/status_im/protocol/web3/delivery.cljs function, message will be sent if:

  1. Wasn’t previously successfully sent via shh.posts(during current session), we retry to do this max-attempts-number * 5 times
  2. Was successfully sent, but requires ack which wasn’t received
    • + previous attempt to send message (during current session) was done more than ack-not-received-s-interval seconds ago
    • + we did not more than max-attempts-number attempts previously (during current session)

Keys

Key type asym (asymmetric) means that asymmetric key will be used on whisper side to encrypt message before sending, in our case it's always user's public key, this means that currently messages with asym key type are supposed to be sent from UserA to UserB directly, but not to any group of users.

Key type sym (symmetric) means that symmetric key will be used to encrypt message. Currently all symkeys are generated from passwords which are hardcoded or are groups' ids, so this is not secure and must be changed.


Type

Description

Key type

:message

1-1 chat message

asym

:group-message

group message

sym (group's id)

:public-group-message

public group message

sym (group's id)

:ack

confirms message delivery

asym

:seen

confirms that message was seen

asym

:clock-value-request

when you send a message and you’re offline,

we send :clock-value-request to obtain the information

about the :clock-value from sender

to make sure that it’s the same

asym

:clock-value

?

asym

:group-invitation

is sent by admin

asym

:update-group

currently this message is sent when some user is removed from group, in result group's keypair is updated

asym

:add-group-identity

is sent when a new user is added to group

sym (group's id)

:remove-group-identity

is sent when some user is removed from group (by admin), after that :update-group with new keypair is sent

sym (group's id)

:leave-group

is sent when user leaves group

sym (group's id)

:contact-request

-

asym

:discover

sym

:discoveries-request

sym

:discoveries-response

sym

:profile

sym

:update-keys

sym

:online

sym


:message

{:message-id    "1498723691404-7711c1b3-411c-5e4b-a64b-dc0ab2317c5a",
:requires-ack? true,
:type          :message,
:timestamp     1498723691404,
:content       "123",
:content-type  "text/plain",
:clock-value   1,
:show?         true}

:group-message

{:message-id    "1498724252836-24753a20-b67e-50ae-8fcd-95b22a1ee78b",
:group-id      "1498723975430-5a3f18f5-189e-5966-b159-bdf837d47e58",
:content       "{\"iv\":[2071556246,-1120403611,-1962404966,921696242],\"v\":1,\"iter\":1000,\"ks\":128,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"ct\":[-1840979203,1955718739,-1431291835,8794767622144],\"tag\":\"75b9b052a2c2835d8ae9525bd6ae3245e879f4b4b8b510fca22b4ad142a2f10913a83c6ff87e80ef766e194d5017cd1eca6c27fb41611cf9c8d8cf3a5df8980b6959d6d66bbfc9e4cf8a6af27decc1663f60a9e201992236a9ea21b2ccab7284\"}",
:username      nil,
:type          :group-message,
:show?         true,
:clock-value   1,
:requires-ack? true,
:content-type  "text/plain",
:timestamp     1498724252865}

:public-group-message

{:message-id    "1498724827175-9c097aa0-d859-5ddd-bdd1-0c83e52d930e",
:group-id      "wow",
:content       "123",
:username      "Name",
:type          :public-group-message,
:show?         true,
:clock-value   1,
:requires-ack? false,
:content-type  "text/plain",
:timestamp     1498724827202}

:contact-request

{:message-id    "1498723171063-c3ecb16a-b3fd-5ee1-99f0-783ef7a1070c",
:requires-ack? true,
:type          :contact-request,
:contact       {:name          "Yearly Onerlooked Westafricanantelope",
                :profile-image "..."},
;; public key is used by UserA (who sends this request) for broadcasting changes of his profile to UserB 
;; (and other contacts) 
:keypair       {:public  "..",
                :private ".."},
:content       nil}

:online

{:message-id    "1498723263025-a83a4fe0-eff6-518d-9348-2570d1494067",
:requires-ack? false,
:type          :online,
:content       "{\"iv\":[1966114636,-1876596685,-393541684,-299376444],\"v\":1,\"iter\":1000,\"ks\":128,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"ct\":[-1468463324,357257435,1338099857,1642960690,-1068289456,705936716,-1541701378,-565102533,26389113252352],\"tag\":\"3151bd11093fdeb90ccdf311b660386b2edb959a5c52d9ed63095fb9ac9bf3d9bdb7ce3841dce1d2820c04d43ca2548c171c4049e06e498b35e6bea3e210afd8dfb32437663e1497a8d3ed7772b72160edc69f124e69e2385ca19cfc5692c7f9\"}"}

:profile

{:message-id "1498723799459-6642807d-2b32-58cd-98d6-19359ad3d8da",
:type       :profile,
:timestamp  1498723799465,
:content    "encrypted content: username, status, photo etc"}

:group-invitation

{:group-admin   "0x04e40d8b966a6644bb8ef2a5c1e170c7c0ef9ba83e8d0cb2e2f07930d092af51cdec3aab1c97b1c451460d0e3525fd5b079ab32ba1ff179fc0db2f047ddb71a703",
:message-id    "1498723975437-09f7ca06-2ffe-5067-87d0-a65d128efb31",
:contacts      ["0x0428c9d6c1aaaa8369a7c63819684f30e34396dc0907d49afeac85a0a774ccb919b3482097d992e66bcc538e7a0c6acf874c77748f396f53c0a102e10d1a37765b"
               "0x04e40d8b966a6644bb8ef2a5c1e170c7c0ef9ba83e8d0cb2e2f07930d092af51cdec3aab1c97b1c451460d0e3525fd5b079ab32ba1ff179fc0db2f047ddb71a703"],
:group-id      "1498723975430-5a3f18f5-189e-5966-b159-bdf837d47e58",
:content       nil,
:type          :group-invitation,
:requires-ack? true,
:group-name    "T",
;; this keypair is used for sending messages to all group members, so that any user could encrypt/decrypt message
:keypair       {:public  "..",
               :private ".."},
:timestamp     1498723975439}

:discoveries-request

{:message-id "1498724283011-65da6bd7-1cb0-566f-a055-27d66cbd04e4",
:type       :discoveries-request,
:content    nil}

:remove-group-dentity

{:message-id    "1498724483015-0d468376-69f5-551c-a6c1-3762f7cb9651",
:requires-ack? true,
:type          :remove-group-identity,
:identity      "id",
:group-id      "1498723975430-5a3f18f5-189e-5966-b159-bdf837d47e58",
:username      nil,
:timestamp     1498724483015,
:content       nil}

:update-group

{:group-admin   "0x04e40d8b966a6644bb8ef2a5c1e170c7c0ef9ba83e8d0cb2e2f07930d092af51cdec3aab1c97b1c451460d0e3525fd5b079ab32ba1ff179fc0db2f047ddb71a703",
:message-id    "1498725030208-c44f66bb-4d9c-5022-b992-eb663511792a",
:contacts      ("0x04e40d8b966a6644bb8ef2a5c1e170c7c0ef9ba83e8d0cb2e2f07930d092af51cdec3aab1c97b1c451460d0e3525fd5b079ab32ba1ff179fc0db2f047ddb71a703"              "0x0428c9d6c1aaaa8369a7c63819684f30e34396dc0907d49afeac85a0a774ccb919b3482097d992e66bcc538e7a0c6acf874c77748f396f53c0a102e10d1a37765b"),
:group-id      "1498723975430-5a3f18f5-189e-5966-b159-bdf837d47e58",
:content       nil,
:type          :update-group,
:requires-ack? true,
:group-name    "T",
:keypair       {:private "..", :public  ".."},
:timestamp     1498725030220}

:add-group-identity

{:message-id    "1498725030221-3a4f2890-3897-546b-8819-c240d99089a3",
:requires-ack? true,
:type          :add-group-identity,
:identity      "0x0428c9d6c1aaaa8369a7c63819684f30e34396dc0907d49afeac85a0a774ccb919b3482097d992e66bcc538e7a0c6acf874c77748f396f53c0a102e10d1a37765b", :group-id "1498723975430-5a3f18f5-189e-5966-b159-bdf837d47e58",
:username      nil,
:timestamp     1498725030222,
:content       nil}

:seen

{:message-id    "1498724483032-ef9f6889-5699-5603-9ffd-05bee945f146",
:requires-ack? false,
:type          :seen,
:group-id      "1498723975430-5a3f18f5-189e-5966-b159-bdf837d47e58",
:content       nil}
{:message-id    "1498725866330-4a77d7d9-5d18-5628-8cf4-1f7d862f5122",
:requires-ack? false,
:type          :seen,
:group-id      nil,
:content       nil}

:leave-group

{:message-id    "1498725404895-36369ea8-1403-5554-b921-c58aaa8d1e0a",
:requires-ack? true,
:type          :leave-group,
:group-id      "1498723975430-5a3f18f5-189e-5966-b159-bdf837d47e58",
:username      nil,
:timestamp     1498725404895,
:content       nil}

:discover

{:message-id "1498725562980-10a58c17-b994-5dd3-87b1-cbdc30533df0",
:type       :discover,
:status     "the #life we're given is on a thread, so wear it well",
:hashtags   ["life"],
:profile    {:name          "Name",
            :profile-image "data:image/png;base64,..."},
:content    nil}

:discoveries-response

{:message-id "1498725718717-2c483805-da99-568b-8958-1e9a54d30630",
:type       :discoveries-response,
:data       [{:message-id "1498725562980-10a58c17-b994-5dd3-87b1-cbdc30533df",
             :name       "Name",
             :status     "the #life we're given is on a thread",
             :whisper-id "0x04e40d8b966a6644bb8ef2a5c1e170c7c0ef9ba83e8d0cb2e2f07930d092af51cdec3aab1c97b1c451460d0e3525fd5b079ab32ba1ff179fc0db2f047ddb71a703",
             :photo-path "data:image/png;base64,...",
             :tags       ({:name  "life",
                           :count 1}),
             :created-at 1498725562990}],
:content    nil}

:ack

{:message-id     "1498725867070-78e60ec0-266e-5dba-a9f6-e53a333f38c7",
:type           :message,
:ack?           true,
:ack-of-message "1498725866330-4a77d7d9-5d18-5628-8cf4-1f7d862f5122",
:group-id       nil,
:content        nil}