Data Visualization in Drupal

does anyone know if it's supposed to

turn green or is red so well thank you

for very much for coming here it's

really awesome to see this many people

coming for a data visualization

discussion it's my first Drupal

conference presentation so I'm a little

bit nervous but I'm excited to share

with you the charts module so just to

give an introduction to myself I'm

Daniel Katherine and you may know me

online as aunty Waco

I work at a public health company called

John snow incorporated or JSI and we've

been around for over 40 years and have

implemented public health projects and

over a hundred different countries I

started with JSI coming out of public

health school and I fully intended to do

public health and now I'm leading a team

of Drupal developers and building and

maintaining Drupal sites if I had to fit

myself into a Drupal archetype I would

probably say I am a site builder

however since around 2016 I've started

creating or taking over the maintenance

of some modules charts is definitely one

of my favorites as it ties in neatly

with some of the work that I do at JSI

for example just yesterday I launched a

website in DC that relies heavily on the

charts module I'm not a season or a

particularly adept programmer

programming jargon is still somewhat of

a foreign language to me but through my

tenacity and commitment to Drupal I

think I've done a lot to add value to

the community and I I asked for a lot of

help and I wanted to give a shout out to

a couple of my colleagues and mentors

Michael Moyes a and Elias maleeni who

both sort of set me down the path to

figuring out how to port the charts

module to Drupal 8 so here's the site I

launched yesterday so you can see we

have a number of different visualization

is going on there so the outline of my

presentation is I'm gonna be talking

about when you might want to charkh will

give an introduction to the charts

module then talk about how to use it

sort of from the inner user interface

and then I'll step into the code and

discuss the code for a little bit and

then if there aren't too many questions

during the FAQ then we can try to do

some small demo so let's start off by

talking about how data visualization can

add value to a Drupal website so here's

an example that's pretty close to home

the project pages on drupal.org have

these visualizations that show the

number of downloads and you know which

version this is the one for charts

beyond module maintainer x' these

visualizations are useful to juries as

he's writing his juries note they're

also fodder for Twitter trolls trying to

say oh Drupal's not getting enough

traction

whatever else but in general these types

of visualizations are great for

ecommerce sites for marketers for user

interface designers and a number of

other you know groups of people that I

want I'm just not thinking about now

charts are also great for adding value

to content so I used to work at a

medical lab and every week we would have

a journal club and rather than like

going through long passages of text we

tended to just like focus on the graphs

that were included in the articles and

discuss those charts are also great for

concise communication this pie chart

provides a lot of facts about the world

without having an enormous run-on

sentence and it's also eye catching in

addition to providing facts data

visualizations can be greatly putting

the facts in context and charts may

influence the behavior of your website

users so including an HR like this could

increase your users trusts in your

organization and potentially make them

more likely to donate to your nonprofit

I showed you images from news

organizations online publications

nonprofits I could have shown you images

from government websites or educational

institutions these types of

organizations are all known Drupal users

and I think if they want first they want

first-class data

excuse me content management from Drupal

but why not also best of class data

visualization and Drupal without having

to you know go to too many third-party

sources

so I believe if Drupal can you know

incorporate the using core and you know

manipulate data with views that should

also be able to chart it and I'm

definitely not the first person to have

that thought because eleven years ago

the charts module went up on drupal.org

to my knowledge it was posted by a quick

sketch currently around 20,000 sites

report using the charts module though

most of them are still Drupal 7 around

been getting pretty good traction and

although its support it's grown some in

scope so originally there were two

charting providers that you could use

the charts module I've included three

more since i ported the module and

depending on if we do the demo or not we

could even incorporate a fourth one care

amina 54th newman the original libraries

were yeah i already said that these are

the libraries that are currently in the

Drupal 8 module they're all great

libraries that have the documentation

pretty strong online communities some

are more active than other C 3 I've

included here but I feel like billboard

digests is which is a branch or what's

the right word they split off of C 3 and

I think they've sort of modernized it a

little bit and so if anyone was asking

me between those two I would probably

drop them a billboard Jas

there's more than one way to use the

charts and module in my head I kind of

categorize it in two different ways so I

think I think charts really excels at

plotting data from an existing database

whether or not that's your Drupal

database or some external database that

just holds data quick sidenote if you're

connecting to external databases

I recommend the views database connector

that's another module I'm maintaining

you may also want to be adding charts

sort of at the time of content creation

and I'm thinking especially like if

you're using the layout builder and you

know creating a like a long page and you

want to have a nice little chart you

could you know use the Settings tray and

how to block and use the charts block to

incorporate a visualization there so

let's talk about installing and using

charts a quick notice that charts

doesn't work without a sub-module

enabled the sub modules make the

necessary JavaScript libraries available

and getting those actual JavaScript

files into your Drupal site is probably

the biggest barrier to using charts it

doesn't really seem that there's a

consensus among the community about the

best way to install JavaScript libraries

I mean we've pretty much all settled on

composer for PHP libraries

JavaScript it still seems like it's up

in the air in the charts module in the

sub module readme.txt files we include I

think very detailed step-by-step

instructions for being able to use

composer to incorporate JavaScript

libraries into your site which you may

have strong feelings about if you're not

a command-line person but you're

comfortable enough installing modules

you can use the library manager module

and manually add the necessary libraries

so after you enable the module then go

to the configure home settings and

you'll set up the default chart

configurations what this does is

especially if you're using views to

create your charts it just starts you

out with a number of different settings

for example if you want to have data

markers enabled by default or you want

to pick a particular library you could

install multiple sub modules like pie

charts Google cetera but if you want

high charts to be the first option when

someone's creating a new chart then you

can set that on this page I'm in the

process of making it so a little bit

like the appearance page on so like you

can have your global theme settings or

you can have per theme settings

so once you've set those defaults the

next thing to do is create a view and

when you're creating your views that the

the view settings to well you don't have

to do content but make sure that you use

the display format of chart and then

your charting fields

you the first thing you'll need to do

after you have created this view is to

add a field that contains some sort of

numeric data or you can apply

aggregation to get a number and then

click on the the format settings link

and it'll open up a window like you see

and and then there's where you can set

per chart settings about data markers

and what you want to use is your data

field and the chart title and all those

types of things and when you click apply

then you should see a chart in your

preview just like that

so what about if you want to combo chart

meaning a chart that has more than one

chart type to do that you simply add a

chart attachment display it's pretty and

then you can figure that it's pretty

much the same as configuring the parent

display and then you attach it to the

parent and then there are a couple other

settings in the middle of the view where

you can say you know I want a secondary

y-axis axis or inherent contextual

filters and things like that there are

some gotchas that can occur with the

chart attachment and so if anyone out

there is a documentation chemistry

contributor it'd be great to have some

support okay so that's a pretty quick

rundown of using the charts module

through the user interface there one

warned a lot of screens because it is

actually pretty easy to do I'm going to

go through how the module works sort of

on the back end and I hope that it makes

the module a little bit less daunting

and maybe spur some ideas so in general

the charts module can be broken into

core and sub modules many of the sub

modules are charting providers and you

can easily create your own charting

provider that

without having to touch the code of the

church score it doesn't and it doesn't

need to live in the charts module either

there's also an example module to

demonstrate how the API works and a

charts override example module and the

starter kits directory and the next few

slides I'm going to talk about the

source directory okay so the first

directory is annotation and we use

annotation basically to avoid info hooks

they basically allow you to include

metadata in your plugin without having

to have those close the next is the form

directory and this allows you to

configure this is that charts default

settings form that you saw in the

earlier slide it's what creates that

form the plug-in folder is a very

important one the first folder chart you

probably don't really need to look at

unless you're wanting to score some easy

commits and get your three phpstorm

license the style in the views the views

plugins there's three of them one is

style that's kind of the main one it

does a lot of the data transformation

and rendering and that's also sort of

the form that you see when you're

clicking on format settings in your view

does the display plug-in creates the

chart attachment which I mentioned a

second ago and then the field plug-in is

a recent development and it creates

abuse field called a scatter field that

allows you to

select it as your data field and then it

gives you coordinates from two other

fields in your view

the third directory and under plugins is

the override and you know plugins are

it's great with object-oriented

programming you can extend classes but

that means that you might also need to

create a custom style plug-in and it

gets to be a lot of programming and

you'll start to feel like maybe I like

should have just like created my own

custom module why am I using the charts

module but we did want to give people

options for some customizations without

having to extend the class and so what

you can do is in the starter kits

there's a module called charts overrides

and you can just put that into your

custom folder and then basically change

the options that you want based on that

charting provider's API and then that

gets that into the primary plugin for

your charting provider

I think that'll make a little bit more

sense later on

the services director use a few services

to make a module more pluggable there's

a lot of code in these services there's

not a lot of code so the settings

directory has some code to pay attention

to the charts based setting form

basically contains like all of the form

elements that you'll see when you're

configuring a chart whether that's in

views or in charts blocks or on the

default settings form then there's the

charts default colors this is primarily

for getting colors to use in various

places in the module the charts default

settings just has a few basic settings

set up so that it just sort of has

sensible defaults basically and then the

excuse me the charts type class sort of

explains different chart types like bar

versus column and whether or not it has

a single or a double axis

there's very little going on in the the

theme directory it provides an interface

for defining charts constants and

they're just two which are related to

it's like single or double axis the

utility directory is one of the more

important directories in the module even

if it only contains one file it handles

much of the data manipulation and in the

next few slides I'll go through these

police methods so the views data method

transforms the views results into an

array of arrays

you're probably familiar with that if

you work in Drupal within the inner

array or objects that contain some of

the more important information for

charting libraries like type and color

and value the remove unselected fields

method takes out fields that are not

selected by your view as in the data

sort of field set and it ensures that

the for loops performed later on iterate

over the appropriate number of times and

then the create charts data method takes

that array that you saw a second ago and

and then it transforms it into this one

which is sort of broken into what you'll

see in the charts lingo as the

categories array in the series data

array and then lastly the check missing

library method looks to see if you

properly install the JavaScript

libraries for your module and it alerts

users that they may need to install a

charting library if they're at a few and

they're not seeing anything

so the charts module file has a few

different hooks they're tough use data

to identify the scatter field and set up

the data structure for that there's book

theme which doesn't really do a lot

other than sort of points you in the

direction of the correct template file

and then there's a template pre-process

hook which is used for generating charts

by through views it does a lot of heavy

work of combining your parent chart in

your chart attachments and bringing them

all together and it sends the data over

to the utility file to be processed by

those methods so that brings us to the

sub modules there's one major file in

each each of the charting provider sub

modules and that's at source plug-in

chart and then the training providers

named HP and this file makes use of the

classes within the settings directory

which you can see settings like in this

example well charge a s and then

a number of different classes in there

what it does is it creates a JSON object

that is attached to the HTML element and

then it's it's attached as a data

attribute and then that data attribute

is fed to the JavaScript from the

charting provider and your charts

rendered so ideally in that settings

entry you'll have a lot of classes and

they'll match up you know with the

corresponding libraries API so in this

example text goes up for text you know

why matches why vertical-align matches

it's a lot I mean if you've gone to high

charts API for example its enormous and

it's a lot of work to include all this

so I could definitely I would definitely

appreciate any hope I'm just making the

module stronger so speaking of which the

areas where I would really appreciate

people's help are making it easy for

anyone to install charting libraries so

even if there's not great consensus on

the best way to do that I would

appreciate other ideas having a better

user interface for charts blocks which

currently it acts that's you to just

enter in your data like separated by a

comma in a like a text area field so

it's it's not super user it's not super

intuitive

one thing I'd really like our dynamic

options when you're like in the views

settings form for example and you select

one particular library if there are

additional like settings that like for

example c3 doesn't have 3d charts so you

wouldn't need to see a checkbox that

says like make this chart 3d but for

high charts you can do that so that

would be useful form element to have

writing tests is always appreciated and

then manual testing and feedback and as

always documentation it's really

important so that's sort of the first

part of my presentation and I'd love to

go ahead and answer any questions that

you have

watch the examples yeah definitely good

is this integrated which is like The

O'Jays honey you guys would have

it I haven't integrated it with JSON API

and I honestly don't really know how to

do that so if you have a decoupled site

I assume you would sort of used the API

but I don't know sort of what your next

step would be it

I like I said I'm a site builder and I

all of my sites are sort of traditional

Drupal sites that are coupled just yeah

yeah I mean you can you can reach out to

me through like the drupal contact form

yeah find me i'm on twitter is dan for

qol or or just create an issue in this

uq but yeah I'm the only Drupal 8

maintainer for charts so and I'm like

after this presentation I'm happy to sit

with you

now that's just query carries this how

close to dad can you like custom queries

because I think that you know it's

almost thinking like if you set the Sun

to it an e-commerce site and sit there

polling charts of you know products it's

selling fast but it's not selling the

best that kind of things and so how do

you pulling like a query statement

the way I would do is just with use yeah

so I pretty much only third the charts

API example has a controller but the

only times I've used the module really

are for using views because you know

it's views is so powerful for that kind

of thing and you know it'll it enables a

lot of like other classes of you know

users to create visualizations so yeah I

think that's probably how I would

approach it

any other questions how many people have

used the Drupal 8 version of charts

good yeah yeah okay so all good was

later how many people are sort of PHP

programmers okay so let me start off

with I'm just doing the user interface

okay

perhaps I can see this bitter so I have

a charts demo content site set up and

[Music]

find my mouse okay

so I'm going to create a new a new sharp

good of use i i've already enabled the

charts module and I've enabled some of

the sub modules as well so out of you

let's get that right now so I have a

couple content types that have data

fields attached to them I'm going to

choose award because a little bit and

I'll create a page with

sharp as the display format showing as I

mentioned fields we can just start with

this is not the fastest computer but uh

okay so I have the title already in the

fields and I'm going to add value which

I know is a field on this content type

I'm can have it as unformatted because

sometimes commas can mess up like the

the library's want to see a number not a

string with the kana so I'm gonna use

unformatted

and then so here's the important part

this settings this is this is what the

style plug-in creates and so I'll start

out with a column for a charting library

and it started with high charts and then

we have title and here's the section

where you do your which field provides

data so I'm going to select this sorry

it's hard for me to see and then

where's my mouse

[Music]

enable tooltips

with the legend position bottom

something like that

so ah you have your church

so I mentioned if you want a combo chart

you can do that by adding a chart

attachment so

all right so it's complaining at least

one data field must be selected I'm

going to add a different data field so

that it makes sense and I'm going to do

that through a relationship

so I know that this content type of Ward

has taxonomy afield for location and the

location taxonomy has a feel for

population

and let me add that population field

and so now let's go to the settings and

instead of a column I'd like this to be

aligned and

[Music]

here greens fine for me when you have a

chart attachment you don't really need

to configure these because these all get

inherited by the the settings on the

parent display so you can pretty much

just ignore this for a chart attachment

okay so you can see there's a line graph

here now and that's just showing what

you get in a chart attachment but now if

you come to the page

oh sorry I forgot

so this was what is mentioning there's a

few chart settings you need to apply so

I'm just gonna say attach onto the page

display and then this is this is above I

haven't fixed yet but I need I want to

keep it here but I just need to click

apply ok so now I'm going to the page

excuse me when you first started to

chart attachment view what display form

did you use was it was a chart

attachment a patient chart attachment is

a display type and the dial plug-in is

set to chart automatically okay so you

still have to configure the settings for

it

but it yeah okay so now you can see you

have your combo chart and you can you

know use contextual filters you can use

exposed filters so when you you know

those typical things you would do with

use you can do with charts the other

thing that you can do and I hope it

works is for example let's do some three

yeah so now you have a c3 chart with

only that one change and you could do

that for any of them any questions there

is the chart attachment now also using

no

but because pretty much all the settings

except for the label field and data

field are inherited it doesn't really

matter what you put here says you

override

for something like that I mean you can

only have one chart type for chart you

know for a chart so that you wouldn't

really want to override you can have

multiple different chart charting

providers on the same page so you can do

one view as a block on a page and then

you could do another view as a block and

put them on the same page one could be

high charts one it would be c3

the other thing I could show you is I'll

show you how the charts blocks look I'm

really not very happy with them but you

there's a time and place for them and

and there's some level of functionality

there okay so I have two that I've

already set up so that these if you want

to add one what you do is just click

place block and then select charts block

and you get that same form pretty much

as you had four views there's a few

modifications so I'll do a column again

I'll do hi charts again I'm going to do

a single series

yeah

[Music]

and then you have the normal sort of

charcoal form here at the bottom

so these are the other two the added to

this page

so I'm not sure if I did something wrong

leave yours did I not do this slash

between categories okay

and I should have kept them this one a

better name

[Music]

if you would have been pulling data from

a different database which box did you

put together and change

yeah so the trucks blocks doesn't have a

good way of incorporating data from

another database in that case I can show

you let me just try to get this it

that's yeah yeah of course

yeah

so there's the doomed

let me go back to views real quick

so I i've also enabled views database

connector on this site and with that it

actually allows you to use tables from

an external database so like i know that

this one has data that i could what

and and it's it just you add them as

fields it's pretty much the same as a

content type okay so maybe i can show

you all code since there were a number

of PHP programmers

[Music]

so I wanted to demonstrate how easy it

is to add another charting provider so

if you're not happy with hi charts or

Google charts III etc you can add your

own and I was talking to someone

recently and they mentioned Vega light

do any of you know that charting library

no it's not whatever

it's not moving over

okay the Kennington here

yeah

okay so what I did is

and maybe let me show you

[Music]

so this is big alight

has a ton of different options so they

find more than most others I've seen and

I've never worked with it before

yesterday what I did was I went to the

charts modules and I copied charts

underscore charts jeaious and I just

pasted it into my custom folder and then

I just went through and I just renamed

anytime chart Jes showed up I replaced

it with big alight so here's here's

everything that's there so libraries I'm

doing all I did is I did I cheated a

little bit because I'm using remote

libraries we're mostly with the charts

you go and use local ones so I didn't

really do much other than just add these

here and then

so for the settings I just added a few

that I knew were necessary labor life

and then it was then it was pretty much

all working here and here so

so that I have a chart type bill truck

type method that you can use if the the

names that we have is like the default

chart types like bar everything don't

exactly match up with like Google for

example wants you to have it has a camel

case name for every one of the so you

could you could change that and

something like this and then the build

variables this is sort of the primary

method that's used within these and so

all i've done is i've created a new

theta like chart object and then i've

done a little bit of data manipulation

because it requires you see if i can

pull that up

okay it requires the the data object to

look like this which is not the way that

we've set it up in that data array that

I showed you earlier on in the

presentation so I had to do some mean

manipulation to have within data have

values and then have like the a matches

up with x-axis and B matches up with

y-axis and then you also have to have

this encoding so that's what I some of

those things I set up here like the

encoding I set up here I did a bit of a

little bit of manipulation and I didn't

do the manipulation properly because as

I mentioned I launched a site yesterday

but I all I did is I enabled that module

cleared caches to bring in the libraries

Vega light showed up I selected it and

then when I saved my chart I have a big

light someone who knows the Vega light

API would be able to create this and

probably like an hour or 30 minutes like

it took me a very short amount of time

it was all basically manually going and

renaming you know from charging us to to

take a light and that was like from 70%

of what I had to do and then a little

bit of you know changing the JSON object

structure so

I think that's it unless anyone else has

questions

I mean sorry you would do Cleveland

module within like you're not putting

into the charts module just because of

the way charts was built with like sort

of a core and like sub module setups

like like Drupal itself the the core

module can identify charts plugins that

are in other modules and then making use

of them so yeah so that's nice because

if you just want to use you maybe you

have your own custom visualization

library or something like that you can

just use use that and you can use it in

your custom module you don't have to

worry about making it so neat and

contributing it back to the charts

module yeah

yeah so that's a that's definitely a

best practice probably hi charts and one

of the reasons for that let me go to

this Ida launch day one of the reasons

for that so here's an example of the

scatterplot is it gives you this little

hamburger menu and you can you can

download as a image if you want save it

into a Word document or something which

is awesome you can also start you've got

really small you can view data table and

so it gives the data table and that's as

far as I've heard is much better for

accessibility than more firm yeah yeah

so so that's probably my favorite the

problem is is its proprietary so you

either have to pay a licensing fee or if

you're a non-profit you can potentially

be exempt from that

yeah

any other Christians

okay I think that's it then I really

appreciate your time

[Applause]