← Back | Deepak Cherian | 03 Nov 2017

bokeh and Emacs org-mode

1. Introduction

I’ve been wanting to play around with interactive Javascript plots for a while; for example: https://demo.bokehplots.com/apps/selection_histogram. The selection functionality is really cool!

This post is an experiment in embedding bokeh plots in org-mode notebooks that rendered properly when exported as HTML pages. There are two approaches for embedding.

2. Using autoload_static (preferred)

Based on https://necromuralist.github.io/data_science/posts/bokeh-test/

Some notes:

  1. This method stores the data in a separate .js file which should make the org file easier to handle. Also, the figure can embedded freely in many places.
  2. Figuring where to get the bokeh css and js files can be tricky (docs):
    1. You can use the CDN: from bokeh.resources import CDN to load the bokeh js file over the internet. Not good for future-proofing.
    2. Using bokeh.resources.Resources(mode='absolute') or 'mode=relative' loads bokeh from your local python installation; so that’s good for local notebooks but your figures/notebooks might break when bokeh updates.
    3. The inline option inlines the necessary js and css in the exported .js file but then with multiple images; you end up with multiple copies so that’s wasteful, but seems like the only future-proof option. Also, you can have two figures made with different bokeh versions embedded in the same document, so that’s a plus.

2.1. Prerequisites

I am using

  1. bokeh 0.12.10
  2. org 9.1.2
  3. emacs 25.3.50.2
  4. ob-ipython @ 1642a74

I define a function export_bokeh to do the actual heavy lifting. This function could be moved to your ipython startup.py file so that it is defined for every ipython session.

def export_bokeh(plot, outPNG, outJS, outHTML, bkjs='inline'):
    from bokeh.io import export_png
    from bokeh.embed import autoload_static, file_html
    import bokeh.resources

    if bkjs is 'local':
        # use local installed bokeh scripts
        wherebokeh = bokeh.resources.Resources(mode='absolute')

    if bkjs is 'remote' or bkjs is 'CDN':
        # use CDN bokeh scripts
        wherebokeh = bokeh.resources.CDN

    if bkjs is 'relative':
        # bkjs is a relative path to the locally installed bokeh files
        wherebokeh = bokeh.resources.Resources(mode='relative', root_dir='./')

    if bkjs is 'inline':
        wherebokeh = bokeh.resources.INLINE

    # save the png file
    export_png(plot, filename=outPNG)

    # save the html file
    html = file_html(plot, wherebokeh, None)
    with open(outHTML, 'w') as file:
        file.write(html)

    js, script = autoload_static(plot, wherebokeh, outJS)

    # save the .js file
    with open(outJS, "w") as writer:
        writer.write(js)

    # embed in the org-exported HTML file
    print('''#+BEGIN_EXPORT html\n{script}\n#+END_EXPORT'''.format(script=script.lstrip()))

Embedding the figure is then quite easy once you

  1. tell ob-ipython to capture stdout and put it in a drawer so that it’s replaced everytime to rerun the source block.
  2. provide a custom filename for export (fname below).

I use this header line:

#+BEGIN_SRC ipython :session :results output drawer :exports both :var fname="my-image" :var titlestr="Demonstrating bokeh"

2.2. Example

This will export a png image, an HTML file and a Javascript file. That way there are both static and dynamic versions that are easy to share as well as an emebeddable version.

from bokeh.plotting import figure
from bokeh.layouts import gridplot
from bokeh.models import ColumnDataSource, HoverTool, WheelZoomTool

tools='box_select, reset'

# create "subplots"
p1 = figure(tools=tools)
p1.background_fill_alpha = 0.0
p1.border_fill_alpha = 0.0

p2 = figure(tools=tools, x_range=p1.x_range, y_range=p1.y_range)
p2.background_fill_alpha = 0.0
p2.border_fill_alpha = 0.0

# generate data
x = np.random.randn(200)
y0 = x**3 - 100
y1 = x**2 - 100

# generate random letters
import string
import random
label = [random.choice(string.ascii_letters) for aa in range(200)]

# needed for linked selection
source = ColumnDataSource(data={'x': x, 'y0': y0, 'y1': y1,
                                'label': label})

# plot the data
p1.circle('x', 'y0', source=source)
p2.scatter('x', 'y1', source=source)

p1.title.text = titlestr

wheel = WheelZoomTool()
# add some tooltips
hover = HoverTool()
hover.tooltips = [
    ("(x,y)", "($x, $y)"),  # '$' for co-ordinates
    ("label", "@label")  # note '@' for column lookup
]
p1.add_tools(hover, wheel)
p2.add_tools(hover, wheel)

p1.toolbar.active_scroll = wheel
p2.toolbar.active_scroll = wheel

# layout the subplots
pg = gridplot([[p1, p2]], plot_width=400, plot_height=400)

export_bokeh(pg,
             '../static/'+fname+'.png',
             '../static/js/'+fname+'.js',
             '../static/html/'+fname+'.html')

The above figure is a rendering of the RESULTS block:

#+RESULTS:
:RESULTS:
#+BEGIN_EXPORT html
<script
    src="../static/js/my-image.js"
    id="f1c145ed-260c-411c-8c26-f71b52427218"
    data-bokeh-model-id="5f0145de-1c4e-4ee2-9a64-f38f255c81ff"
    data-bokeh-doc-id="20261111-dbef-4dd0-b40c-d8c04db088bd"
></script>
#+END_EXPORT
:END:

2.3. Improvements

Things I would like:

  • Automatically run the HTML export when running the code block.
    • Can I add-hook to org-babel-execute-maybe?
  • The ability to auto-hide the javascript in the RESULTS drawer but show the exported png file.
    • Ideally, I would see the png in the org file and the javascript in the HTML file. The other should be hidden. Seems too complicated.
    • A nice solution would be to insert a link to the png file in the image caption.
  • Are actual captions (#+CAPTION) possible with bokeh figures?
  • Avoid the explicit export_bokeh call if possible.
    • Ideally, ob-ipython would be able to tell that this is a bokeh block; use the fname var to pass the figure handle p and fname to export_bokeh and do everything. This would be close to jupyter notebook workflow.
    • This might be possible using IPython formatters as suggested by the ob-ipython README.
      • Could we then define export_bokeh in imports.py and call that when the returned object is a bokeh figure handle?
  • Can we get holoviews to work with this? Optionally tell it to use either matplotlib or bokeh backends

3. Using components (not-recommended)

I made some slight modifications to http://kitchingroup.cheme.cmu.edu/blog/2016/02/07/Interactive-Bokeh-plots-in-HTML/ though I didn’t need to embed anything in a frame. The principal drawback here is that all the data is returned in stdout and written to the org file. This would get unwieldy for big plots.

  1. Add the following at the top of my org file to use locally downloaded versions of bokeh. Note that versions need to match/be compatible with what you have installed (bokeh.__version__).

       #+HTML_HEAD: <link rel="stylesheet" href="/home/deepak/tools/bokeh/bokeh-0.12.10.min.css" type="text/css" />
       #+HTML_HEAD: <link rel="stylesheet" href="/home/deepak/tools/bokeh/bokeh-widgets-0.12.10.min.css" type="text/css" />
       #+HTML_HEAD: <script type="text/javascript" src="/home/deepak/tools/bokeh/bokeh-0.12.10.min.js"></script>
       #+HTML_HEAD: <script type="text/javascript" src="/home/deepak/tools/bokeh/bokeh-widgets-0.12.10.min.js"></script>
    
  2. For this website I use the CDN urls they provide : http://cdn.pydata.org/bokeh/release/bokeh-x.y.z.min.css etc.
def WriteToOrg(p):
    script, div = components(p)

    script = '\n'.join(['#+HTML_HEAD_EXTRA: ' + line for line in script.lstrip().split('\n')])

    print('''{script}\n#+BEGIN_EXPORT html\n{div}\n#+END_EXPORT'''.format(script=script, div=div))


from bokeh.plotting import figure
from bokeh.embed import components

p = figure()
p.line(np.random.randn(200), np.random.randn(200))
WriteToOrg(p)

4. Appendix

4.1. yasnippet

A simplified version of the one on ob-ipython. This autogenerates a random filename.

#+BEGIN_SRC ipython :session :results output drawer :exports both :var fname=${1:`(make-temp-name "img")`}
from bokeh.plotting import figure
from bokeh.layouts import gridplot
from bokeh.models import ColumnDataSource, HoverTool, WheelZoomTool

tools='box_select, reset'

# create "subplots"
hf = figure(tools=tools)
hf.background_fill_alpha = 0.0
hf.border_fill_alpha = 0.0

hf.$0

export_bokeh(hf, 'images/'+fname+'.png', 'images/js/'+fname+'.js', bkjs='inline')
#+END_SRC

4.2. Dependencies

Looks like exporting to PNG requires

conda install phantomjs selenium

4.3. Styling

  1. Either include a css file or add a style tag to HTML_HEAD at the top of your org file.

       #+HTML_HEAD: <style> add-stuff-here </style>
    
  2. Targeting the css class bk-plot-layout lets you center the image. bk-grid-column does it for gridplot

       .bk-plot-layout .bk-grid-column { max-width: 100%; margin: auto;}
    

4.4. Source

Here is org-mode source for this page.

Creative Commons Attribution License Creative Commons Attribution License

This work is licensed under a Creative Commons Attribution 4.0 International License.