Amazing Emails from Drupal 8

- [Man] You just wanna run back

through your slides, and then just?

- [Wayne] I want my
four minutes, at the end.

(laughing)
Or two minutes, okay.

- [Man] Now just go
back to where you were.

- [Wayne] Wait, so you don't want me

to say everything again?

Okay, all right, so we were here.

All right, we're gonna talk about

the Website Editor Experience.

So you have staff people
who work on your website,

and they would would like to be able

to edit the content of
transactional emails

or know what transactional emails occurred

from your website, and with Drupal,

or at least with most Drupal sites,

that's not particularly easy.

It's certainly not easy out of the box.

Specifically out of the box,

there's no HTML formatting
of Drupal emails.

There are limited features of emails

out of Drupal, so it doesn't
support attachments or images.

The editor experience can be inconsistent.

So if you're on your site,
you're working with Drupal Core,

oh, you wanna edit those account emails,

that's under Account Settings

or Configuration of Account Settings.

You have no WYSIWYG editor, no HTML,

and this is what the user
interface looks like.

You install Webform.

It has a completely different interface

for handling emails that come from it.

It's actually really great.

It gives you HTML.

You can do Twig Templates.

But it's just in a different
place, and it looks different.

If you have Drupal Commerce,
better fire up PHP Storm

because the only way to edit
that receipt is in a Twig file.

Another issue with emails from Drupal

is there's no email history.

So you have no way to know

what Drupal sent

from your website,

which leads to my most favorite message

that I get from clients, which is,

"So-and-so says they didn't
get that reminder email."

Or "they didn't get the
password reset email."

And we know it's in the SPAM folder

or whatever, but we can't
prove it, so then we get this.

So if we look at sort of a report card

of the email capabilities of
Drupal, it's mostly incomplete.

We don't really have HTML formatting,

advance features means
attachments or images,

we don't really have that.

The editor experience is
like, okay, not so great.

And email history, we
don't have that either.

So this talk is gonna be about
how to fix all these problems

and hopefully have a better report card

for emails from Drupal at the end.

All right, so we're gonna
make Drupal emails amazing.

All right, little bit
about me, my name's Wayne.

I'm an independent Drupal
developer for a long time.

I split my time between consulting

and development work under Zengenuity.

Which is also my handle
on Drupal.org and Slack.

And so I split my time between that,

and I also run training
courses at DrupalTutor.com.

So specifically what
we're gonna talk about

in this presentation is how email works.

We're gonna talk a little
bit about how Drupal works.

I'm gonna show you the architecture

of a better email solution for Drupal 8.

And then I'm gonna--

the big reveal will be the
new system that I built

to make this really easy to do
amazing emails from Drupal 8.

This presentation is
targeted at site builders.

So I'm not gonna go
deep into a lot of code.

And so those of you who are programmers,

some of this is gonna
be kind of rudimentary,

but I have targeted
this for a site builder.

One thing we are gonna talk
about is email deliverability.

If you're having trouble with
your messages going to SPAM,

this talk is not gonna help you.

The only thing I will say
is use a third party MTA,

don't send emails from your web server.

Or they're definitely going to SPAM.

All right, so how email works.

Email is just a text
file, or a text stream,

if it's going over the internet.

So here's an example of the
source code of an email.

The most important line
in this is this one.

The first new line break
in this text document

separates the email headers
from the email body.

The email headers are
metadata about the email,

some of which you will
see in your email client,

like the subject or the recipient,

who it was addressed to.

Some of them are more technical.

These are ones you wouldn't
normally see unless

you looked at the source
code of the email.

This is a really important one right here

and specifically for this talk,

there is a header in
your email that specifies

what is the content type
of the body of this email.

So this is an example
of a plain text email.

So the content type says text/plain,

and then the body, you can see,

doesn't have any HTML tags in it.

But that header is what
tells your email client

that it should not render this--

that it should render this as plain text.

This is an example of an HTML email.

It is structured basically the same

except, uh, the content type is text/html,

and then the body includes
HTML formatting, that--

and this header will
tell your email client

this should be rendered as
though it were like in a browser.

If you have a email that includes both

an HTML and a plain text version,

then it gets a little
bit more complicated.

The content type is multi-part,
multi-part alternative,

and there is this other, like, long string

of characters that's the boundary line.

That boundary line is
how this text document

is later separated into pieces.

So that boundary line at
the beginning of that,

there's another set of
headers, except for, like,

the plain text version at the beginning

and then the HTML section below.

So then your email client can decide,

"Do I want to show HTML?"

Maybe you've said, "I
never want to see HTML.

"Please show me the plain text version,"

in which case, your email client

can choose to just show the one.

Attachments work pretty much the same way.

They are multi-part mixed,

but they have one of
these boundary separators,

and then headers that identify this

as, like, an image, this is a PNG image,

it says it should be
treated as an attachment,

and then that image is encoded

as text in this text document.

So if you ever wonder when
you get a big attachment,

and your email client seems,
like, really slow to load,

or in general why email clients

are the slowest thing on your computer?

It's because they have
to parse, or at least

with new emails, they have to parse this

text document and turn it into

all those files that are attached,

everybody's cool images that

they have for their signatures and stuff.

If you have images in an email,

there's a couple different
ways that that can be done.

Most of the time, that's done with a URL

to a remote server that has
that image sitting on it.

So this is just like standard HTML

with a link to a server somewhere else.

The image here is not
attached to the email at all.

It depends on the remote server.

There is one other way to do that, though.

With inline images you can
have a hidden attachment

that has a content ID,
and then that content ID

can be used as the source,

and if the image is like this,

you won't see it
typically as an attachment

on the email, but it can be used

in the email format, you know in the HTML.

A few notes about HTML email.

It is limited HTML and CSS Implementation,

so all that really,
like, cool parallax stuff

that flies out, you
can't do that in email.

Responsive design does
work, for the most part,

in most popular clients.

There's limited support for web fonts

in the most popular clients.

Gmail and Apple Mail, both Mac and iOS,

good support for most of
the advanced features.

Outlook is horrible.

If you have to do Outlook emails,

you have to transport yourself back

to, like, at least 15 or 20 years,

where you're doing a table-based layout,

and it's really awful.

Now luckily, if you look at, this is

last month's numbers
for email client share.

If we add Gmail and Apple Mail's together,

that's like 72% of people,
so that's pretty good.

Now if you're in a B to B environment,

probably you're gonna have more Outlook,

so tough, sorry.

If you talk about how Drupal send emails,

this is basically the totality
of Drupal's email API.

It has three major functions.

There is a mail-manager service

that has a mail function.

It takes a parameter,
a module that should be

sending this mail, a
key, which is a unique ID

for one type of mail,

uh, the recipient, and then
some other parameters...

The main one there being the params,

which is an array of other data

that should be used for
customizing this email.

The module, who is responsible
for sending this email

or generating email
content--they're not technically

doing the send--needs
to implement hook mail.

And this, uh, when somebody calls

the first method up
there, Drupal will call

their hook mail implementation
because it knows

which module based on the other call.

It will pass the key a
message, which is an object

that has some, it's already preset

most of the basic stuff, and then you

can add the body and the subject.

If it's a text-based email,

you might just have the
text there in the code.

If it's an HTML email, you might pass back

a render array that has a Twig template.

And then hook mail alter
is if you have a module

that wants to change the
email that's being sent

by another module, like
if I want to change

the Drupal commerce receipt,
then I can implement

hook mail alter in my custom module,

and I can just manipulate
it however I want,

redo it, throw out what
they did, do anything.

So that's pretty much it.

By default the Drupal mail-manager service

only supports plain text.

So all that multi-part stuff we saw?

Doesn't know anything about it.

Coming back to our, uh, our report card.

Now we know the technical side

of what we need to accomplish.

The first two items that are missing here:

the HTML formatting and
the advance features,

all relate to that whole
multi-part sending issue,

or the lack of it.

And so let's solve that first.

Luckily, that's easily solvable

by something that already exists.

There's Swift Mailer system.

It's actually a PHP library.

It's part of the symphony project.

And it's designed specifically

for sending multi-part emails.

It provides both the formatting,

meaning it can generate those text

with all the boundaries
and all that stuff,

and transport, it can, you can decide,

"I would like to send this
email using SMTP server

versus the on web server PHP."

The library can handle that as well.

Swift Mailer support
attachments and inline images,

and there's a Drupal integration module

that you can install, that
will add all these features.

In order to use Swift Mailer on Drupal,

you also need to install
the mail-system module.

The mail-system module
gives you a user interface

for selecting which processor

you would like to use for email.

So it allows you to select
one for the sender--

Sorry the formatter and the sender,

and it also allows you to select which

theme should be used when
rendering HTML emails.

You could also do that
selection per module,

and even per unique email key.

So you can go in here and say,

"I want the account emails since they

"were originally in plain text,

"I just want to leave them in plain text,

"but the commerce order receipt

"should be sent with HTML,"

which means it needs Swift Mailer.

So you could make the
formatter, in that case,

Swift Mailer just for
commerce if you want.

If you want to customize
Swift Mailer, uh because--

Uh actually, once you get to this point,

you've got HTML email, but the HTML

is very boring; all right it's
very simple, but it is HTML.

So probably you're
gonna wanna customize it

to give it your brand
colors and possibly fonts.

There is a Twig template
that comes with this mailer

that you could override
in a theme to customize

the wrapper around the
HTML for your HTML emails.

It's pretty simple, it just has sort of

a single-body variable that's, like,

all the stuff that's
supposed to go in email.

And you can see it has a
very conservative design.

It's using a table-based approach

uh, so that it will be compatible

with Outlook by default,
which is another reason

you might wanna customize this.

It also has a very simple style sheet

with it in the template, that just sets

the font, size, and family.

So you will almost certainly wanna

customize that if you want to introduce

your brand colors or other elements.

That's also where you can do,

if you want to do, responsive
design in the email.

Media queries can go in here.

So if you have some kind of column system

that you wanna have collapse,

you can do that by just
replacing this style

or adding to this style in your overridden

version of Swift Mailer HTML Twig.

That's also where you can do web fonts.

So you can get this sort
of font face definition.

This is actually Google Fonts.

They work great, actually, at least in

Apple Mail and Gmail.

And so you can just put those in there.

Obviously be somewhat careful of that.

If you put, like, a
million of them in there,

people have to download all these fonts

in order to render the page.

But with what we get with Swift Mailer,

we can make pretty emails.

All right, we can have nice formatting.

We don't have to look at that giant

text string for the reset password.

We can just have a Reset
Your Password button

that looks nice.

We can have header, brand color, logos.

All that stuff's possible
just at this point

now that we've installed Swift Mailer

mail system and customized the template.

(mumbles)

So if we look back at our report card,

we made some progress.

HTML formatting, definitely
got it, looks great.

Advanced features, so they are supported,

but there's no user interface for them.

So if you go into code, if
you wanted to, let's say,

add an attachment to an email,

it's possible now that you've installed

Swift Mailer, but there's not interface

for you to do it.

It can only be done in code.

And that's more or less true for other

images and things you might
want to imbed as well.

The editor experience has
probably taken a step down,

or at least I put it that way
because we still have this,

and now that we support HTML emails,

this we're gonna have to type
the HTML email ourselves,

and we have no CK editor
or any of the text formats

or any of the things
we would normally have

if we were writing a
note or something else

on the Drupal site.

So we now actually require people,

at this point, to know HTML.

Um, so...

- [Audience Member] I
was just wondering...

So, like, if you try Swift
Mailer for the whole thing,

then those emails all
show up as just one line,

is that right?

- [Wayne] They used to,
but they don't anymore.

There's something in
there that prevents it.

- [Audience Member]
Okay. Well, that's good.

- [Wayne] Yeah.

All right, so let's move
on to email history.

So in order to do email
history, we're gonna have to--

If we wanna know, store what
happened on the Drupal--

what emails we sent, then we're gonna

have to store it in the Drupal database.

So we need to know a little bit about

how Drupal stores data,
stores data as entities.

Entities are data records.

They have different types, so there's

content, users, taxonomy terms.

There are entity bundles,
which are subtypes.

All right, so the difference
between an article

and a blog post--

Oh, sorry, a blog post
and a page or something

would be the bundle type of the entity.

And the entity system also supports...

Oops. I think I missed...

Oh, so if we're gonna store our email,

then we're gonna need
probably some kind of email,

we're gonna need an entity
type to store this log

of emails we've sent,
and the different bundles

can represent our
different email templates

because each bundle can have
a different configuration.

And then the individual email
history log will be entities.

If we use the entity system,
we'll also have access

to tokens, which is great because we need

a templating system, and
tokens is great for that.

Drupal already did that pretty well.

So these are the
components we're gonna need

in order to create um, to create,

um, an amazing Drupal system.

We're gonna need templates.

We're gonna need some way to create them.

We're gonna need a way to store them,

a way to trigger an email from there,

and then these other things, the actual

transport, we already talked about

that we have a mail system,
it's Swift Mailer, for.

So, okay I'm just gonna briefly say this

cause I'm gonna be low on time.

My initial solution to this
was the message module.

Has anyone in here used
message module before?

Okay, yeah, so message
module's a generic framework

for storing messages to
users in your system.

Um, so I use that, and that basically

took care of the entity parts.

But I had some problems.

First you have to sorta hack
into the Drupal mail system

every time to override
its normal functioning.

Like if you send a
password reset, you hafta

then generate a message entity

and send it yourself, so you have to

write code for every template.

The template experience is not great.

This is the form; it's not really labeled,

it's like subject and body,
even though that's what it is

because it's a generic
framework for messages.

It does not specifically for email.

The email history doesn't
really have formatting.

Because, again, it's not only for emails,

so when you look at the message log,

it doesn't look so--

That's not exactly what it looked like

when it was sent to this user.

So it's better.

It does give us email
history; it's, like, okay.

The editor experience is like mainly

a little daunting and confusing.

But it does work, and I ran with this

on DrupalTutor.com for at least a year.

Since I was a Drupal
guy, it wasn't a big deal

about the template issue.

But then I need to
build this for a client,

and so I needed something
that would be better

or simpler, particularly for them to use.

So my new solution needs
to be optimized for email,

needs to eliminate the
requirement for custom code,

the templates should be easily
editable and previewable

by content admins, and
they should have access

to all the important features,

like attachments and images and stuff.

So this new system is available now

as Easy Email, the Contrib module.

Okay, so it's designed
to be as easy as possible

to set up, as easy as possible to use.

My web form, sort of my,
you know, model for this

it should be really
easy for site builders.

So if we look back at this diagram.

Easy Email provides it's own entity type.

It does all the storage,
and for the creation trigger

there's this email overrides module,

which I'll show you in a minute

so that you can...

You don't have to write to, like,

intercept the normal email
and do something different.

All the custom design
work, that's all pushed

to Swift Mailer, so
everything I showed you

at the beginning about Swift Mailer,

is all still valid.

That's how you put in your
CSS from your templates.

Easy Email does add a few extra templates

for wrappers for the HTML body,

the plain text body,
and the inbox preview.

Inbox preview is this text that you see

before you click open the email.

Normally that is whatever
is at the beginning

of the email, but if you do a little trick

of a hidden div at the
top, you can customize

that text so that it
doesn't actually have to be

the same as what's in the email.

All right, this is what the
template editor experience

looks like, the page, the
form; it looks like real email.

Okay you have normal email fields

that people expect: To, CC, BCC,

all the From-related fields,

the standard fields.

You've got the CK editor
for rich text on the HTML,

so any of the sort of
plug ins, if you've got

uploads for images,
linking, all that stuff's

built in here.

Nobody has to know HTML
in order to edit this.

It supports tokens, of course.

And it supports attachments so you can

add in either pass here and
this also works with tokens.

If you have some sort of
file attached to an entity

that's somehow referenced here, you can

dynamically attach those
to the outgoing emails.

Supports plain text alternative.

Templates are fieldable,
so you can add fields

that you need for the special things,

for each template, so if it's
the commerce order receipt,

then you will need a
reference to a commerce order

so you can that reference field.

It has configurable base fields.

These are not technically base fields,

there bundle fields, but when you create

a new template, it creates all these

extra fields on here, so let's say

you don't want to do plain text?

You can just delete this field,

and it will go away from the interface,

and it will go away from the database

if it's all of them.

It has user reference fields built in

that will automatically track.

So when you send an email
to a specific address,

it will look up to see if that matches

a user account on the system,

reference that, and
save it in the history,

which allows you then to create a view

so you can say to search
through the history

by user account, or you can create a view

that's like a tab on
the user profile page.

It's like, "What are all the
emails we sent to this user?"

Click. Boom, they're all listed out.

Template preview: so, the, uh, this--

When you are creating a template,

you can preview it, and it shows you

what it will actually
look like as an email

both in the inbox,

and has this what the HTML body

and the plain text body look like.

It has a little dragger right here,

and I don't think I'm gonna have time

for the demos, but if you
can drag this back and forth,

and then, if you set
up a responsive design

you can actually see how it's
gonna collapse in that design.

It has duplicate email prevention.

So this something that I
needed for DrupalTutor.com.

I have webinars, I send
out reminders, sometimes

people have more than
one class registration

that might generate a webinar reminder.

I don't want to send
dupes, so you can create

with tokens the unique pattern,

which then will prevent duplicate emails

that match that same
pattern from being sent out.

The email override system, this allows--

this is basically a user interface

for hook mail alter.

It allows you to select the
email you want to override,

select the replacement template,

and then do parameter mapping.

So like password reset
has an account parameter,

it needs to be mapped to
some field on the template

so that you can use the correct tokens.

So if you were overriding
a commerce receipt,

you would also map the original commerce

parameter object to some field,

to a reference field that you

would have added to that template.

It support ad-hoc sending, so if you need

to send somebody a password reset

cause they can't find
the link on the website,

you can just send them one.

Of course, it has email history

that's more tailored to emails.

That email history
includes a full-text view

of what it, what we really sent this user,

including the HTML.

Since it uses the entity API,

it works with views.

It uses all the...

It has all the

entity hooks and the things
that you would expect.

All the templates and overrides
exportable configuration.

Has a test suite, not
totally complete yet,

but the basic sending
and template previewing

creation all have tests.

I am using this on a couple of site

in production; seems fine.

Future roadmap, these are
things I'd like to add.

Choose a theme per template.

It'd be nice if you could change the theme

for each of your templates.

So if you want to have
one that not as branded

cause it's like that
special email from the CEO,

who, like, when you
sign up for the service

and it needs to look like
it came from his iPhone,

then, like, you know, you
could change the theme.

Better inline image handling.

So I showed you there's two ways to do it.

It'd be nice--

It currently only supports
the one with the URL

to the remote server.

It would be nice if there was a check box,

and then it would do the other one.

Default bundle field: that's just like

if you don't want all those fields

that you have to remove
to have it default.

Disable email saving per template.

Right now it saves all of them,

which on high-volume sites is probably

gonna be way too much data, so it would be

nice to not save password
resets or whatever.

As well as the option to purge older ones.

Message bundle actually has that,

so you could probably just import it over.

And then web form integration.

So, now our report card's
looking a lot better.

We've got HTML formatting.

We have advanced features
like images and attachments.

And they're easily
accessible to the editors.

The editors, they have
full control over all

of the content that goes out
in those transactional emails.

And the email history's gonna stop people

from asking us if the website's down,

or the email's broken or whatever.

A couple example emails
that I've built with this.

So this is that webinar reminder email

I was talking about.

So, again, we have tokens
that go in to evaluate

for what the webinar URL
is and the date, time.

This is one that I built
especially for the talk,

specifically for the talk, but it's based

on one that the client
that I was working with

was using, which is they
wanted to be able to add

like an up-sale message,
like an upcoming-event

message to the bottom of all

the commerce receipts that
were sent from their site.

So this is an example of how that worked.

They can basically just go in and edit

the receipt template and
do whatever they want

down in this area.

So they can upload images,
they can type in whatever

they want, links, et cetera.

This is one we did, but we
haven't fully implemented it yet.

Uh, QR codes because this is
for an event-registration site.

QR codes that people can scan.

This is done with a custom token.

And I think that's all my time.

I do have demos, but we're
not gonna have time for them.

So, but yeah, so that's it.

Easy Email; if you're interested in it,

I would love to have some other people

contribute patches for
any of those features,

or sponsor development.

I am a mercenary for hire,
so yeah, so talk to me

if you are interested in that system.

(applause)

- [Audience Member] That was a standing O.

- [Wayne] (Laughs) Okay. Thank you.

- [Audience Member] I have
one question will quick.

- [Wayne] Yeah.

- [Audience Member] Do you think--

so the email overrides, did you do it

with config entity or content entity?

- [Wayne] Config.

- [Audience Member] Sweet,
so you could say, like,

out of the box, I'm gonna
override four emails.

- [Wayne] Yes.

- [Audience Member] And route
them, cause I would love

to, like, make them be part of, like,

we have a (mumbles)
demo we do for commerce.

I'd love to just like have
this be part of this demo.

Cause I could see saying,
"Commerce, here's your email,

blah, blah, blah, blah, blah."

- [Wayne] Yeah. Yeah, yeah.

- [Audience Member] This is
slick, this is really cool.

- [Man] This is really good.

- [Attendee] How did you...

When you use, like, custom tokens

how do you deal with that on preview?

- [Wayne] So the tokens,
well, so the custom tokens,

you know there's an API where
you create those in code.

Those will just be evaluated
just like all the other tokens.

There's no difference
in ones that you write

in your custom module and
ones that come from core.

Assuming, obviously that you're following

the way you're supposed to do tokens.

- [Attendee] Yeah, I'm
just curious, so you're

doing the template, so, like, how do--

- [Wayne] So this QR
code, like what they see

on the editor's side, they just see

the message, that short-form format

that's like a QR code to the border,

and literally it's like a
QR code to the border ID,

or border URL.

- [Attendee] Does Easy
Email take, does it have--

Like Swift Mail has the
dependency of mail systems--

- [Wayne] Yeah.

- Does this have--
- Yeah.

- a dependency--
- Yeah, this all requires

Swift Mailer and mail systems
because it's offloading

all the technical stuff
to those two modules.

This is really like, this
is like the user interface

for Swift Mailer and Mail System for that.

(mumbles)

- [Attendee] How does this work
with send rate loads, like--

- [Wayne] So if you, so
Swift Mailer supports SMTP,

so it is the same way if
you can use the SMTP option,

then you could just configure that in the

Swift Mailer configuration page onsite.

(mumbles)

- [Attendee] Nice.

- [Audience member] Wonderful.

- [Attendee] Awesome.

- [Wayne] Thanks.

Captions made possible by ClarityPartners.com. Chicago area Drupal consultants