Refreshing computer-skills: markdown, mathjax, jupyter, docker, microkernels

8 June 2016 Roberto Innocente - inno@sissa.it 1
Refreshing computer skills
1) How to easily write simple Web pages ?
2) How to easily add math to Web pages ?
3) Notebooks : mixing text, math and results of computations
4) News on virtualization
5) News on Windows Server 2016
6) News on Microsoft Windows 10
7) Scaling out

1
How to easily write simple Web pages ?
Using the markdown language,
or a markdown WYSIWYG editor like
remarkable (linux), markdownpad (windows),
typora (windows), markdown_edit

Markdown
● HTML (Hyper Text Mark Up Language) is the official language of the Web
(T.Berners-Lee 1991)
● In the last decade many thought that HTML is too complicate for writing simple
pages. So in opposition to it, a very simple language named markdown was
developed for writing Web pages by humans.
● This was initially implemented as a perl program markdown.pl that is a
markdown→html translator
● Then it was implemented in python as markdown_py module
● Now many WYSIWYG editors are available for all the platforms. These editors
can usually also apply a set of CSS (Style Sheets) to the final HTML document.

markdown
========
is a text-to-html conversion tool developed by john gruber in perl.
[original project](http://daringfireball.net/projects/markdown/)
[original text](http://daringfireball.net/projects/markdown/index.text)
(there is an implementation in php too)
####python-markdown
is a python library that implements gruber's markdown.
[python markdown library page](https://pypi.python.org/pypi/Markdown)
There are some differences between the python library implementation
and gruber's tool.
[differences](https://pythonhosted.org/Markdown/index.html#differences)
####python command line script
[python command line (not on the anaconda python) :](https://pythonhosted.org/Markdown/cli.html)
$ /usr/bin/python -m markdown -x markdown.extensions.tables [options] [args]
$ /usr/bin/python -m markdown -x markdown.extensions.tables markdown.txt
>markdown.html
python names the script markdown_py to avoid confusion with the perl one.
$ /usr/bin/python -m markdown input_file.txt
Piping input and output (on STDIN and STDOUT) is fully supported as well. For example:
$ echo "Some **Markdown** text." | /usr/bin/python -m markdown > output.html
Use the --help option for a list all available options and arguments:
$ /usr/bin/python -m markdown --help
You can make text *italic* or **bold**
You can build **bulleted **or **numbered** lists :
remarkable SolarizedLight CSS

* alpha
- alpha,alpha
- alpha,beta
* beta
- beta, alpha
----
1. alpha
+ alpha,beta
+ alpha beta
+ beta
you can add an horizontal rule :
---
make **blockquotes** :
>very difficult very difficult very difficult very difficult
>to explain very difficult very difficult very difficult
>>all this very difficult very difficult very difficult
>>and the rest of it very difficult very difficult very difficult
you can insert links :
[sissa homepage](http://www.sissa.it)
you can make headings up to level 5 :
#heading 1
##heading 2
###heading 3
####heading 4
remarkable CSS github

#####heading 5
you can insert code to display (prepending a tab) :
if (i<10){
printf("%d",n);
} else {
printf("%s","stop it !");
}
def f(x):
return x + 1
github markdown for code (triple backticks) :
---
```python
print("Hello world!")
```
---
```javascript
var s = "Javascript" ;
alert(s);
```
---
Insert tables in a very simple way :
(Warning : with simple markdown the tables have no borders, in jupyter they have borders), it depends on css
|Header|Header|Header|Header|
|------|------|------|------|
|Cell |Cell |Cell | Cell |
|Cell |Cell |Cell | Cell |
|Cell |Cell |Cell | Cell|
---
remarkable CSS markdown

Markdown | Less | Pretty
--- | --- | ---
*Still* | `renders` | **nicely**
1 | 2 | 3
---
markdown is a superset of html so you can add html tags :
<b>wow</b>
---
![alt text](https://www.sissa.it/sites/all/themes/sissa/logo.png ":Logo:")
you can refer to local files :
![/home/inno/]ws2016-Dockerfile
Section Title
=============
Subsection Title
----------------
Inline `code` has `backticks` around it.
remarkable CSS dark

Python Markdown Editor
This is an open source editor written in python and based on a small
httpd server using ajax for refreshing the rendering. It works on all
platforms (requires python).
● Of course you need Pyhton installed
● And you need pip : the python package manager.
● Type :
– pip install markdowneditor
● pip

2
How to easily add math to Web pages ?
Using mathjax

LaTeX, MathML, OpenMath
● Most of the people in science that need to write math, since more
than 25 years, use LaTeX. At the beginning of the web people that
needed to display math on web pages were using latex2html that
was displaying the picture of the math formula rendered by LaTeX.
● Around ~2000 the W3C consortium approved independently from
this wide community of users of math, a standard for writing math
on the Web called MathML (Mathematical Markup Language, its
version 3 is now part of HTML5 and became the ISO standard
40314 in 2015)

MathML splits presentation (presentation
MathML) from semantics (content MathML)
and so 2 different parts are required !!!!
The simple formula
Requires all that you see on the left pane
OpenMath tries to define instead only the
semantics of math in a user-extensible way
and so is partly overlapped with content
MathML. sin(x) in OpenMath :
<OMOBJ>
  <OMA>
    <OMS name="sin" cd="transc1"/>
    <OMV name="x"/>
  </OMA>
</OMOBJ>
Presentation MathML
Content MathML

MathJax
● Of course no one of these solutions was satisfactory for many
● In 2009 , with funding from AMS (American Mathematical Society) and
SIAM , a JavaScript program called mathJax was developed, that could
replace formulas in LaTeX inside the Web pages with their rendering made
using CSS and SVG
● This was a revolution .. now everyone can insert math in web pages using
the largely diffused LaTeX syntax, just inserting a link to this JavaScript
program. And these formulas are scalable (because of SVG) and scale like
all the rest of the page.
● is now simply $ax^2+bx+c$

MathJax/2
● It is easy to use mathJax : you just need to include in the
header of your html the link to the script (showed in next page)
● Many markdown WYSIWYG editor do this automatically and
therefore you can directly write into them LaTex formulas:
– Inline math : using as delimter 1 dollar sign $x^2+1$
– Displayed math : using as delimiters 2 dollar signs
● $$int_{-infty}^y{f(x) dx}$$

<!DOCTYPE html>
<html>
<head>
<title>MathJax TeX Test Page</title>
  <! Copyright (c) 20102013 The MathJax Consortium >
  <meta httpequiv="ContentType" content="text/html; charset=UTF8" />
  <meta httpequiv="XUACompatible" content="IE=edge" />
  <script type="text/javascript" async
       src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeXAMS_HTMLfull">
  </script>
  <script type="text/xmathjaxconfig">
     MathJax.Hub.Config({tex2jax:{inlineMath: [["$","$"],["(",")"]]}});
  </script>
</head>
…
…
</html>
Header for MathJax processing
Of LateX formulas

MathJax
In the html header (this will work also with markdown, because it lets
the html tags to flow trough without problems) you should put :
<script type="text/javascript" async
src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML-full">
</script>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
tex2jax: {inlineMath: [["$","$"],["(",")"]]}
});
</script>
the second script is necessary only if you want to use the standard inline math
delimiters of Latex $ $.
then you can insert your math in 2 ways as inline math :
look at this for instance $x=sum_0^n{(i^2+1)}$, that stays amid the text,
or in displayed math that goes centered on a new line :
$$f(y) = int_{-infty}^{y}{g(x) dx}$$
not the full latex or ams math is supported, look at
[mathjax tex](http://mathjax.readthedocs.org/en/latest/tex.html)
[tex commands supported in mathjax](http://www.onemathematicalcat.org/MathJaxDocumentation/TeXSyntax.ht
m)

3
Notebooks :
mixing text, math, computing results
Using jupyter

In the good old times there was IPhyton (an interactive
version of python).
In time it grew so much to contain many things :
●
An interactive shell
●
A notebook format
●
A notebook conversion tool
●
A client/server protocol
●
An interactive parallel python
Pieces like the notebook and protocol are only
incidentally related to python and not specific of python.
So the great split of Project Jupyter 2014. In august
2015 with Ipyhton 4.0 most of the components passed to
the jupyter project and other projects like ipyparallel,...
Jupyter
“Jupyter is like IPython but language
agnostic”.
The most interesting part of it are
the notebooks. Available in
Mathematica(TM) since many
decades, these are now
implemented in an open source
context and with the possibility to
use many different kernels :
bash, python, maxima, octave, R, ...

Jupyter
Notebooks are organized in cells : a unit of input/output with the background kernel.
Cells can be :
● Markdown cells : in which you put text and eventually math formulas using the markdown language and
mathjax (that is called automatically by jupyter)
● Code cells : in which you put code for your kernel
A cell is entered typing : Shift+Enter.
Markdown cells will be rewritten with their html rendering, code cells will get an output from the kernel (that
in the case of maxima will be in TeX and rewritten by mathJax with a picture of the formula).
Try :
http://try.jupyter.org/
They have bash, haskell, julia, python, R, Ruby, Scala kernels.
You can save your notebooks as .ipynb (notebooks files) or export them as .pdf

Jupyter with maxima kernel

4
News on virtualization
Using Docker and friends ..

Technology convergence
Containers
Docker
Minimalist OSs
New network
protocols
Microservices
Container
orchestration

Microservices
● Opposite to monolithic app.
Develop a single application
as a set of small independent
services (processes)
communicating each other
only trough a lightweight
mechanism (like an http API)
● Microservices are language
and tool independent
Monolitic App
Microservice 2
Microservice 1
Microservice 3
Pic from eugenedvorkin.com

Containers
●
Traditional Virtual Machines are based on
hypervisors that present a bare machine to
the upper layers of software (performance
penalty)
● On Linux since a decade kernel modifications
were adopted to provide os-level (or light)
virtualization : there is only one kernel
shared by all virtual environments and
containers are insulated groups of processes
with regards to resources and namespaces
(pid,fs,net,ipc) (very good performance)
● In Windows Server 2016 Microsoft provides
Windows containers.

Docker
● Linux containers are provided augmenting standard system
calls and are difficult to use (lxc is not a fancy implementation)
● In March 2013 Docker (opensource) took the world by storm …
● Written in golang provides an easy and complete interface for
containers : image creation, image repositories use, running,
stopping, removing of containers ..
● In WS2016 microsoft supports Docker for its containers.

PID, user, groups namespaces
File system namespace
Network namespace
Inside a container : namespace insulation

Docker cheatsheet

Minimalist OSs
● In the container era streamlined OSs are getting revenge on their large opponents.
●
Nothing decorative, only the essential
●
Many different offers: coreOS, rancherOS, atomic(RedHat), snappy (Ubuntu),
boot2docker (docker.com) ..
●
The most used is coreOS (opensource). Public since October 2013 it has :
– Atomic upgrades (uses FastPatch double partition scheme)
– No package manager (all services installed in containers)
– Docker support
– Some cluster support tools : etcd, fleet, flannel, …
– All in all 200MB !

New network protocols
● Hundreds of containers per server = many
thousands of containers and MACs/IPs per rack
●
Impossible to manage them in a standard layer 2
LAN. Very large and expensive switches have only
64k or quite less maximum MAC entries (cisco
4900M ~55k entries, cisco 3560 ~6k entries, alcatel
OS6850 ~8k entries)
● But we want to keep the simplicity of migrating the
containers everywhere in the Data Center :
– Encapsulation/decapsulation by TOR (Top Of Rack)
switches or nodes :
●
MAC-in-MAC
●
MAC-in-IP/UDP
●
Overlay networks : DOVE (IBM), VxLAN(arista,
cisco, vmware), GRE (cisco)
Pic from M.Muktharov slideshare.net

Container orchestration
● It is very useful to schedule containers based on load. Move containers from node to node,
start them up, stop and remove them when it is needed
● The container ecosystem has different (opensource) offers for this :
– Kubernetes (google), swarm (docker), apache/mesos, ..
● Kubernetes (descendent of borg, the tool google used to manage its infrastructure) was
released to the public in july 2015
– It runs pods : one or more containers that are guaranteed to be scheduled on the same machine ( to be
able to share resources)
– A service is made by a set of pods that work together (as one tier in a multi tier app)
– It is appropriate also for very large installations

5
News on
Windows Server 2016
Using containers, docker

Windows Server 2016
● A full Windows container implementation. They can be
managed by powershell commands or docker (included)
● Windows containers are of 2 kinds :
– Normal containers (analogue to Linux containers)
– Hyper-V containers (in this case they are isolated in a VM)
● A windowsservercore image for use with containers
● A nanoserver image (streamlined windows server), again part
of the container ecosystem)

6
News on Windows 10
Using ubuntu bash, Hyper-V

Windows 10
● The anniversary release of Windows 10 to be released in June
will have some new features:
– An ubuntu bash. This is not just a toy. Using paravirtualization,
canonical and microsoft have provided a near to complete
environment in which you can apt-get many working linux tools (eg
emacs)
– Hyper-V of Windows Server fame.The new release of Docker for
windows uses this specific virtualization instead of virtualbox.
– Hyper-V containers

7
Scaling out

Microservices
can easily scale out
Monolithic applications need to scale up
Much more expensive and inflexible

Multi-tier microservices
scaled out
Load
Balancer
(ngnx)
Load
Balancer
(ngnx)
WebApp
WebApp
WebApp
Galera
MariaDB
cluster
Galera
MariaDB
cluster
WebApp
Load
Balancer
(ngnx)
WebApp
Galera
MariaDB
cluster
Automatic
Scaling out

8
More on Docker

Docker flow
Official
Repository
of images
Dockerhub.com
Docker CLI
Command Language
Interface
Docker daemon
Port 2375

Dockerfile/1
# base image debian
FROM debian
MAINTAINER inno@sissa.it
# aptget some tools
RUN apt update && apt install curl
# copy URL, very useful cmd
RUN curl O http://people.sissa.it/~inno/hello
RUN chmod a+x hello
CMD ./hello

Dockerfile/2
# alpine : based on busybox linux
# the “swiss army knife of embedded systems”
FROM alpine
# apk is the aptget of other distributions
RUN apk update && apk add nodejs
RUN mkdir avg
ADD avg.js avg/
WORKDIR avg
# this command will run after the machine is up
ENTRYPOINT[“node”,”avg.js”]

Union file system
● Used by Docker to use images :
– For every line in the docker file a new image is created, they are all
read-only
FROM alpine
RUN apk update && apk ..
RUN mkdir avg
ADD avg.js avg/
Docker r+w layer
You can
Make changes
Writing to the
Last layer
You can
Commit
Changes
To a new
image
New
image
ENTRYPOINT[“node”,”avg.js”]
WORKDIR avg

Project

Project: Coreos Cluster for μservices
Pic from coreOS

Refreshing computer-skills: markdown, mathjax, jupyter, docker, microkernels

Recommandé

Recommandé

Contenu connexe

Similaire à Refreshing computer-skills: markdown, mathjax, jupyter, docker, microkernels

Similaire à Refreshing computer-skills: markdown, mathjax, jupyter, docker, microkernels (20)

Plus de rinnocente

Plus de rinnocente (16)

Dernier

Dernier (20)

Refreshing computer-skills: markdown, mathjax, jupyter, docker, microkernels