Building Web front end for Python scripts with Flask

featured

Recently I revived my relationship with Python in an effort to beat routine tasks appearing here and there. So I started to write some pocket scripts and, luckily, was not the only one on this battlefield – my colleagues also have a bunch of useful scripts. With all those pieces of code sent in email, cloned from repos, grabbed on network shares I started to wonder how much easier would it be if someone aggregated all of them, made a Web UI and shared this experience.

Thus, I started to build web front-end to python scripts with these goals in mind:

  • allow people with zero python knowledge to use the scripts by interaction through simple Web UI;
  • make script’s output more readable by leveraging modern CSS and HTML formatting;
  • aggregate all the scripts in one repo but in a separate sandboxed directories to increase code manageability.

This short demo should give you some taste of what it is:

Disclaimer: I am nowhere near even a junior python or web developer. And what makes matters worse is that I used (a lot) very dangerous coding paradigm – SDD – Stackoverflow Driven Development. So, hurt me plenty if you see some awful mistakes.

An interaction model of PLAZA (this is the name of this project) is straightforward. A user opens a web page, selects a script from the menu and fills in necessary input data.

By hitting submit data goes to the back-end part, where selected python script does it’s noble job and produces some output data. This output data pushes back to the browser and got rendered to the user.

Obviously, one will need some front-end technologies to build the web layer and some back-end to process incoming data.

Tools & Technologies

Front-end

To build a fresh-looking, dynamic web view we need a modern framework. I used Bootstrap package (CSS and JS) as it is well documented and have tons of implementations. What comes good with Bootstrap – JQuery, of course. JQuery was used to handle AJAX response/request messages between front-end and back-end without reloading the whole page. Since I had no previous experience with both of these technologies, I heavily used everything google served me. Here is my list of useful resources I found noteworthy:

  1. Layoutit.com – there you can create Bootstrap grid and play with elements in a drag and drop fashion. Load the result in a zip file and your grid system is almost ready.
  2. Bootply.com – visual builder for Bootstrap layout. It has some good examples which cover basic Bootstrap elements behavior (navbar, grid rules, etc).
  3. Form validator by 1000hz – well, it’s a form validator. And since every script needs to get input data from a user, form validation is a must-have for a sleek user experience.
  4. Bootsnipp.com – crowdsource collection of snippets written with Bootstrap. I grabbed my side menu from it. Another useful section from this site is Form Builder.
  5. Formden – another form builder.

Back-end

The dirty job in the back is done by gorgeous Flask, which is a micro framework for writing web applications. It includes a web-server, Jinja2 templating engine and lots of features to make back-end easy even for dummies like me.

Regarding Flask I used a lot of google, stack overflow and these useful resources:

  1. Famous Flask Mega Tutorial by Miguel Grinberg
  2. Discover Flask – A full season of youtube videos from Michael Herman
  3. Official documentation of course!
  4. Using AJAX requests with Flask from codehandbook.com
  5. Another good post on AJAX+Flask interaction from giantflyingsaucer.com

Project structure overview

This was my general tool set, now it’s time to establish project’s high-level structure.

Flask maintains a simple yet flexible project structure. In my case, I didn’t go much far away from a basic setup, since overall simplicity was also one of the goals.

Although comments above give enough information about the structure, let’s go into details a bit

  1. Flask application – app.py – is an entry point for the whole project. It starts the web-server, loads the routes  (aka links to the pages of your web project) and plugs in python scripts stored in the scripts_bank directory.
  2. As every other app, Flask app should be configured differently for development and production. This is done via the config.py and the environment variables .env file.
  3. In the static directory you normally store your CSS, JS, pictures, custom fonts. So did I.
  4. HTML pages are in the templates directory.
  5. And the pythonic scripts with all the relevant files (unique HTML templates for input forms, additional front-end Javascript code, etc) are living inside the scripts_bank directory.

Configuring Flask

Once you have Flask installed and got familiar with Flask basics (either through official quick start guide or tons of tutorials) it is time to configure it. There are several ways to configure Flask application. The basic one is to specify configuration statements as arguments to your app instance:

A bit more advanced way is to specify all the config parameters in uppercase in your app.py and tell the app instance to get config from this file:

But the methods discussed so far can’t let you have different configurations for Dev and Prod environments (which you’d want to have eventually).

When I was choosing the configuration method for this app I followed a path which consists of three key points:

  1. creating configuration classes for different environments and using inheritance (explained here)
  2. choosing the right configuration class based on the current value of the  environment variable
  3. storing environment variables in a file (.env) and parsing its contents for parameters (dmore here)
Detailed explanation of Flask app configuration

Setting up front-end

Good, Flask app has been configured and is ready to render some pages, so let’s go and prepare out front-end to display projects’ web pages. Download Bootstrap, JQuery, Fontawesome and store theirs minified min.css and min.js files in the static directory. This is how it should look:

What’s your layout?

Before diving into HTML it is advised to think about pages layout. I recommend you to get familiar with Bootstrap CSS rules and choose a layout that fits your project. I decided to go with a 3+9 scheme. Three parts are for side menu and nine parts are for a content area with a navigation bar at the top of the page.

I composed a sketch of the page depicting how I would like to see my projects web view for an arbitrary script:

Follow the link to see my script’s page template on codepen and see how things interact. Do not worry if you can’t pick rock solid layout right now, you will be able to modify it on-the-fly and decide what suits your needs better.

Flask routes & templates

Routes

Flask uses routes to create URL’s for web pages. If we need to show the main page for example for the URL abc.com we need to define the root route –  / – like this:

This will effectively bind function index() to the route / , so when a user navigates to the application root it will trigger index() function execution.

In this project I have just the same situation:

My index() function does one simple thing, it asks Flask to render specific template – index.html. And what is a template?

Templates

You might guess that a template has to do something with HTML content rendered by a browser. Yes, it has, but it is far more powerful than a static HTML file:

Generating HTML from within Python is not fun, and actually pretty cumbersome because you have to do the HTML escaping on your own to keep the application secure. Because of that Flask configures the Jinja2 template engine for you automatically.

So Flask’s template is a Jinja2-based template which allows you to build dynamic web-pages instead of static content. To render a template you can use the render_template() method. All you have to do is provide the name of the template and the variables you want to pass to the template engine.

You can name your templates as you like, but generally it has .html extension to reflect their purpose. This is my index.html template mentioned earlier bound to the route / .
And this is how it’s got rendered:

Dynamic version of the index page can be found on codepen. The trick behind that magic template->rendered page transformation is in the first two lines. This is template inheritance –  {% extends 'base.html' %} – and that is what makes templating so powerful.

Template inheritance

Inheritance drill described briefly in the official documentation and the main part of it sounds like this:

Template inheritance allows you to build a base “skeleton” template that contains all the common elements of your site and defines blocks that child templates can override.

Apart from the official docs, you can watch this video from the “Discover Flask” series to better understand how does template inheritance work?

Main template

One of the best practices regarding template inheritance is to compose a base template or a layout for the whole site so every other template will inherit from it. My “main” template is called base.html and it describes the logical parts for each page in this project.

In other words, the main template consists of static parts like Navbar, side menu, it also connects core CSS, JS and fonts. And finally, it specifies where would child template’s content be placed. 

I marked the lines on which child template insertion occurs. Once again, read the docs on templating, read some blogs and you will catch it quickly.

Child template

Once you have the base template figured out you are ready to create it’s successors – child templates. A while back I showed you /templates/index.html template where the following construct

effectively told Flask to extend base.html content section with some code relevant to this particular index.html page.

Intermediate templates and multiple inheritances

It is also possible to inherit more that once. See what I did for the pages with actual python scripts input and output forms:

As you will see a bit later – my user-facing scripts’ page has some static sections like Description, Usage, Limitations, Author , etc. Normally, all of these sections will appear on every page thus it would be nice to move all this static and repetitious content to a separate template. That’s how content_template.html was born. In this template I define blocks with names corresponding to the static sections.

The last bit of this puzzle is the template <script_name>.html which extends content_template.html and fills in all the data into blocks defined in his parent template. This template will be spawned each time a new script will be added. In the example below this is a template called get_vmrc_links.html.

Follow this link to view the dynamic template on Codepen.

Blueprints

Another project’s major building block is Blueprint. Blueprints are important and actually making it possible to isolate various scripts in their appropriate sandboxes. And by sandbox I mean separate directory which hosts all the files linked to the script.

Take a look inside scripts_bank directory which will host all the scripts-related files:

It’s the blueprints which allow us to modularize the app by storing some of it’s components in the different directories and still be able to link them up to the main Flask app. See how elegantly JS code along with CSS styles needed only by this particular application get_vmrc_links found their’s place in a separate directory – /scripts_bank/vmware/get_vmrc_links/ !

Creation

To create a Blueprint I placed this code in the get_vmrc_links.py:

When I created a blueprint I defined it’s static_url_path to  /get_vmrc_links/static . But don’t get confused if you don’t see this path, I don’t have it. That is because blueprints can be registered from a specific point and not directly from the project’s root.

Once we have Blueprint created we need to bind it to the route (line 4 in the snippet above). And again the route /get_vmrc_links  will have it’s root at the directory where Blueprint will be registered later.

Registration

To register the blueprint navigate to the main app.py and add the following lines:

Registration is easy! Have you spotted the  url_prefix='/vmware' part? This is the Blueprints root directory I was talking about! So now you can glue the parts in a whole picture.

  1. Blueprint’s root directory is /vmware
  2. It’s static directory path is  /get_vmrc_links/static which turns to  /vmware + /get_vmrc_links/static == /vmware/get_vmrc_links/static
  3. The Flask route  /get_vmrc_links transforms to  /vmware/get_vmrc_links and by following this URL the script’s page will be rendered

Front-end<->back-end data exchange

To pass data back and forth between front-end and back-end we need to:

  1. (@front-end) serialize data from the input elements
  2. (@front-end) pass this data to the back-end
  3. (@back-end) receive data, make calculations, construct a response, send it
  4. (@front-end) receive a response and render it in the output form, handle errors

1. Serializing input data

Serializing is not hard at all. Since it is a front-end’s task it is done by the JS code which is also stored in a separate file unique to this particular script  /scripts_bank/vmware/get_vmrc_links/static/get_vmrc_links.js . This example shows you how you separate one script from another by maintaining all related files in a script’s folder, in this example I’m working on get_vmrc_links script, so all the JS and specific HTML templates are stored under  /scripts_bank/vmware/get_vmrc_links/ directory.

Take a look at  get_vmrc_links.js and pay attention to  $('#submit_form').click(function() . This function handles things occurring on on-click event to the  Submit button.

String  data: $('form').serialize(), produces a string of serialized data with all <form>‘s input elements IDs and their values. Along with serialization task, this JS file contains additional things like showing “Loading” overlay and filling inputs with predefined data from select object.

2. Sending serialized data to the back-end

Serialized data goes via POST method to the back-end via an url you specify.

3. Receiving data by the back-end, processing it

To receive serialized data you need to create a POST requests handler:

To get the contents arrived in POST I queried form data structure of the request object with appropriate keys. form[] object is ImmutableDict data structure which contains all the data received in the POST method:

Once you received your inputs you pass it along to the main function of the chosen script to process. Here I should mention that you have two ways of generating output data:

  1. you could leave it in plain text and wrap it in the appropriate HTML tags with Flask
  2. or you could enclose scripts’ output data in HTML tags during scripts execution process

In this example with the get_vmrc_links.py script I chose the latter option and wrapped the whole output of the script (which normally would have found it’s peace in stdout) with HTML tags:

See these <pre>, <p> and <strong> tags I used? It’d done exactly to get rich formatting.

4. Passing the results back to the front-end

One of the goals of this project was to make script’s output look more readable. Thanks to modern front-end techniques and frameworks you could render whatever/however you like, your skills are the limit. At this time, my scripts produce just some text which I can render in various ways with HTML. But how do I actually pass this data to the front-end engine and in a what form?

I pass it as JSON-formatted structure composed in a two-step process:

  1. Firstly, I collected scripts output data as a dict with the keys representing output data and errors (if any):
  2. Once I have a dict with results and errors to show I use Flask’s jsonify function to represnt my dict as JSON and compose a Response object to pass it further to the front-end:

And that’s it. Now fast forward to the front-end and see how it processes received data:

On a successful return, I check if output has any errors and if it has – put an error message in the #output_div block. If things went smooth I put collected results in this block instead.

Adding new script is easy

1. Create file structure

It’s very easy to add a new script. Walk with me and see how easily I add completely separate script called SAM-O XML API Tester.

At first, I created directories which represent sandbox for the script in a folder dedicated to storing scripts (scripts_bank). As I said, my directory structure follows my side-menu bar, that’s why for the new script called SAM-O_XML_API_Tester I first off created root directory 5620sam and then subdirectory SAM-O_XML_API_Tester. The latter dir will carry all files related to this particular script.

Do not forget to create empty __init.py__ files inside directories of the script to treat folders as python packages.

2. Create HTML

Now it’s user-facing HTML template’s turn. I created sam-o_xml_api_tester.html file in SAM-O_XML_API_Tester/templates dir leveraging sandbox environment. See, this makes individual script management very convenient, one directory stores em all.

Following inheritance model this template inherits markup from the content-template.html. As I explained earlier it makes easier to fill in general text information (such as a name of the script, usage guide, author info, etc). Consider this as static or temporary layout for almost every new script.

3. Create & register a Blueprint

Now it’s time to write few lines for back-end part. Create a python file which will hold blueprint for this script we’ve been adding and back-end activities:

Register it in the main  app.py :

And you are good to go!

How to test PLAZA?

Apart from traditional way of cloning a repo and buidling a virtual environment, you can use a docker container.

What’s next?

Tons of useful things are missing at the moment – no search, no active tags, no login-based system, no tests, etc. I will probably add some of this features later, but you are welcome to suggest, blame, and pull-request. Yeah, the code as is can be grabbed from GitHub.

Roman Dodin

Network engineer at Nokia
Eagerness to learn multiplied by passion to share.
You can reach me at LinkedIn

You Might Also Like