holger krekel <holger(a)merlinux.eu> hat am 28. März 2019 14:09 geschrieben: With approaches 2 and 3 [1], either option is possible. If stored in the DCC DB, mails would have their own table(s). With approach 4, it would be part of the existing messages table in the DB. Instead of always referring to a chat, the ID could then alternately refer to an IMAP folder. Since (as I just looked up) the recipients of a message are stored in the chat, mails would need to store their sender and recipients somewhere else. The chat detection logic wouldn't change. Non-chat mails are not associated with a chat. That's what makes them non-chat mails. They would be associated with an IMAP folder where the user and/or Sieve filter rules placed them. Persistence would be the same as for sending (i.e. optional for 2 and 3, mandatory for 4), except that sending implicitly uses only the "Sent" folder, while received mails can be in any folder. So mail storage would need to remember the folder ID for mails instead of the chat ID for messages. No idea. I would like to have an overview of existing events during a message's life cycle as documentation anyway. Then we can add any events which are missing. Off the top of my head I can think of an event when a mail is moved to another folder, but that's hard to detect when another client does it. Most events are probably identical to or a subset of message events. I also think the handling of Autocrypt by one or more email clients is the main use case of Autocrypt, so it should be straight forward to apply it to an email client and a chat-over-email client. I think my only problem with understanding was that I didn't have in mind that recipients are stored in the chat and not in the individual messages. That's yet another difference in the data models between mails and messages. Which results in yet another complexity for approach 4, which in turn makes me like approaches 2 or 3 even more. Regards, Viktor Pracht -- Open-Xchange GmbH, Olper Huette 5f, D-57462 Olpe AG Siegen, HRB 8718 Geschäftsführer: Frank Hoberg

Hi Victor, all, in general, i agree with holgers suggestions. i try to add some more details: On 30.03.19 at 16:59, holger krekel wrote: yip, i'd also suggest to leave encryption up to deltachat-core; otherwise it would be a hard job to keep the keys and states in sync. sth. as `dc_send_raw(context, recipients, mime, flags)` could do the job. flags may be used eg. to bypass the encryption. due to the alrady merged early-mime-generation, such a function might be rather straigt-forward. `dc_send_raw()` could also return the id of the job added; this way the ui can check the state later and/or the core could emit an event. i suggest to re-use the id column of the jobs table. to avoid the same id appearing just after deletion, however, the flag AUTOINCREMENT should set for the column. i've just done this in https://github.com/deltachat/deltachat-core/pull/641 https://delta.chat/en/2019-03-14-oauth2release#streamlined-contact-requests) maybe we need two callbacks: - one to ask the ui if it wants the messages based upon some header-data (Message-ID, Chat-Version) and maybe the decision that deltachat-core has made - if the ui decided to get the message, the core downloads the message (if not yet done) and passes it to another callback. this could be implemented independingly of the decision the core handles the message or not, so it's up to the ui'as decision not to handle messages that are already handled by the core. sth. as `dc_set_raw_receiver(context, callback_check(context, headers, is_also_handled_by_dc_core), callback_receive(context, mime))` then. note, that the ui already can attach userdata on context-creation and can get it back in the callback using dc_get_userdata(). yip. this is already needed for the folling step: this is the corresponding issue. https://github.com/deltachat/deltachat-core/issues/597 i think this should be targeted before handling the other points and might be a good job to start with. best, bjoern On 30.03.19 at 16:59, holger krekel wrote: > Hi Victor, > > returning to your top-level post, in light of our discussion. > > I suggest to focus first on the issue of sending/receiving "non-dc" > emails. I am putting it in quotes because DC also sometimes > groks/processes non-DC mails that are put to chats, persisted and some > of them are also moved to the DeltaChat folder etc. You can not > conclusively determine from just seeing a single mime-message > that it is a non-dc mail. > > Had some discussions with Bjoern about what's involved from DCC's side > for growing send/receive arbitrary mime-messages. We arrived > at this suggestion for possible next steps: > > > - implement a C-api in deltachat.h that allows to register > a callback which receives incoming decrypted e-mails > that DCC does not consider to be a chat message. > (note: this relates to the "show_emails" option, see > https://delta.chat/en/2019-03-14-oauth2release#streamlined-contact-requests) > > The Message ID and some sync-metadata is persisted with DCC > but not the message body itself. > > > - Note also that in this API design, all Autocrypt processing and > state for all contacts would be managed by DCC -- the UX would > need to get indications (a header) if a mail was encrypted. > > As you know, the devil is in the details but we are willing > to help along issues that you encounter :) > > holger > > > > On Thu, Mar 28, 2019 at 13:19 +0100, Viktor Pracht wrote: >> Hello all, >> >> I have started this discussion in the forum[1], but apparently this mailing list is more appropriate. So, below is my original post, followed by Holger's reply with my answers to that inline. >> >>>> As a pre-requisite for "Make DC a replacement for MUAs?"[2] and hybrid >>>> chat/mail clients in general, the DeltaChat Core library should support >>>> sending and receiving arbitrary MIME mails. >>>> >>>> I see several ways to add the API to DCC, in order from least to most >>>> abstraction: >>>> >>>> 1. Send/receive raw bytes. Not stored in DCC database. >>>> Folders are simply strings representing the full path, i.e. starting >>>> with “INBOX/”. >>>> - pro: least effort for DCC >>>> - con: most effort for DCC clients, duplicate DCC functionality >>>> - con: need to expose more internals to access e2ee keys, etc. > > >>>> 2. New class allowing direct access to libetpan. Optionally stored in DCC >>>> database. >>>> Folders have their own class. >>>> - pro: reduced effort to implement now >>>> - con: more effort for clients on migration away from libetpan >>>> 3. New class with all necessary methods, abstracting libetpan. Optionally >>>> stored in DCC database. >>>> Folders have their own class. >>>> - pro: reduced efforts for clients >>>> - con: more effort to implement now >>>> - con: more effort for DCC to migrate away from libetpan >>>> 4. Extend the existing message class (dc_msg_t). Stored in DCC database. >>>> Folders can use either their own class or extend the existing chat >>>> class (dc_chat_t). >>>> - con: big change to existing APIs, code and DB >>>> - con: gets in the way / increases the learning curve for pure chat >>>> clients >>>> >>>> Just counting the pros and cons, options 2 seems the best so far. But >>>> before we start thinking about implementation detail, what are your >>>> thoughts and ideas about it? Maybe some completely different >>>> implementation ideas? >>> >>> Not sure this is a good forum for discussing what are rather implementation APIs and architecture questions. I’d rather suggest the mailing list. Let me still say a few things :) >>> >>> I suggest to think more about the needed API and not too much about how it’s implemented. There is a lot of tentative/incremental flux that could happen in the next months, regarding moving code from C to Rust etc. And let’s maybe first talk about sending/receiving semi-arbitrary mime-mails, not about new/added IMAP handling. Which C-API would be needed and how would it work/be specified? >> >> There might be a misunderstanding. My list does not contain any implementation details. Quite the opposite: before taking the first incremental step, we should first decide on the direction in which we want to go. And that's where the 4 approaches come in. Except for maybe approaches 3 and 4, the different approaches are not incremental extensions of each other. So we will probably need to decide which way to go, and discuss enough details to make the decision. >> >> For example, the sending half of approach 1 is a single trivial function, because most of the work for that was part of the "early MIME generation" issue. It already technically accomplishes the goal of sending arbitrary MIME mails, except that accomplishing the goal this way is almost useless because it forces clients to do everything on their own, from MIME generation to crypto (see also the cons in the list). >> >> "Semi-arbitrary mime-mails" sounds to me like approach 4: adding functionality to the Message class to send more and more types of mails until it's flexible enough to send any MIME mail. So yes, that's one approach which would make sense. But before we start discussing the implementation, I would like to discuss the pros and cons relative to the other approaches. And maybe somebody can come up with yet another completely different approach. >> >>> How does Autocrypt-handling happen including encryption and decryption? If mime-mails are not tied to a chat at sending time, but to a caller-specified list of recipients, then how are they accessed on the receiving side? >> >> I'm not sure I understand the question. Mails are accessed using a mail client, aka MUA (mail user agent). Why should anything be different if the sender's MUA is using DCC instead of e.g. MailCore 2? >> >> Regarding Autocrypt, that should not be any different than people accessing their Autocrypt-encrypted mails on multiple devices/clients. The beauty of chatting over email is that for the purpose of most standards (here: Autocrypt), DeltaChat is just another MUA. >> >> >> Regards, >> Viktor Pracht >> >> [1] https://support.delta.chat/t/support-normal-mails-in-dcc-for-hybrid-clients… >> [2] https://support.delta.chat/t/discussion-make-dc-a-replacement-for-muas/253 >> >> -- >> Open-Xchange GmbH, Olper Huette 5f, D-57462 Olpe >> AG Siegen, HRB 8718 >> Geschäftsführer: Frank Hoberg >> _______________________________________________ >> delta mailing list -- delta(a)codespeak.net >> To unsubscribe send an email to delta-leave(a)codespeak.net > _______________________________________________ > delta mailing list -- delta(a)codespeak.net > To unsubscribe send an email to delta-leave(a)codespeak.net >