DevEnv Workshop - Lab 01¶
You are now going to practice some of the concepts that were demonstrated in the first part of the workshop on your own cloud hosted virtual machine. This guide provides all the necessary commands for each step. Tasks in this lab guide should be performed in order, as they might depend on files and packages installed beforehand.
Setting Up for Remote Development¶
You should perform this step together with your instructor at the beginning of the session. The goal here is to be able to perform all the subsequent lab tasks inside of a VSCode session running on the lab VM.
Start up your VSCode, go to the Extensions tab, and ensure the "Remote - SSH" extension from Microsoft is installed. If you need to install it, type "Remote" in the search bar up top and locate it in the populated results below (VSCode needs to be able to access the Internet for this).
You will also need a compatible SSH client - the VSCode documentation provides instructions for all of the major platforms here.
With the prerequisites installed, you should now have a green >< sign in the bottom left corner of your VSCode window. You may click on it to open a selection of Remote-SSH actions.
You may also open the command palette (
Ctrl-Shift-P) and typeremote-ssh- watch it filter results as you are typing! For a full list of shortcuts, go to File->Preferences->Keyboard Shortcuts.
First, you need to set up your remote SSH host. Select Remote-SSH: Open Configuration File... and choose the first option, depending on your operating system it will look like path/to/home/folder/.ssh/config. In this file, add a new entry (replacing the IP address):
Save this file and close it. From the Remote-SSH menu, now select the Connect to host... action. In the list that appears, you should find devenv-workshop-vm - select it. A new VSCode window will open, asking you for the password (your instructor would have provided this to you). This window is now fully remote, giving you access to the files on the remote host.
In the menu, open File -> Add Folder to Workspace..., then select lab under the /home/ntc path, and press OK. The remote VSCode window may reopen, asking you to input the password again. You should now see the lab folder on the left-hand side Explorer panel.
Expand the lab folder and open lab_guide.md from within. You may read it like this or use the Markdown: Open Preview option from the Command Palette (Ctrl-Shift-P or F1) to view an HTML render of the file.
If not already open, you may start a terminal inside this window by going to the Terminal->New Terminal menu item.
Congratulations, you are all set up!
Backup Option¶
!!! Only use this if you cannot get VSCode + Remote SSH to work. !!!
You can replicate the same functionality using the following tools:
- your SSH client of choice - PuTTY, SecureCRT etc.
- your code editor of choice - SublimeText3, VSCode, Notepad++, VIM/nano etc.
- a file sync client - CyberDuck or WinSCP
You should try to keep this lab guide and an SSH terminal on the lab VM side by side (or on separate monitors) to help you focus on the lab tasks.
Task 1 - Using Python Remotely¶
Step 1¶
In your terminal, ensure you are in the /home/ntc/lab/ folder. Create a new folder here called webserver and change into it.
ntc devenv-01 ~ $ cd /home/ntc/lab/
ntc devenv-01 ~/lab $ mkdir webserver
ntc devenv-01 ~/lab $ cd webserver/
ntc devenv-01 ~/lab/webserver $
Step 2¶
Create an index.html file with containing the following text: <h1>Hello there!</h1>. You may use VSCode for this or run the command shown below.
Step 3¶
Start a simple python3 webserver. Here you are calling the http.server module directly from the command line, a very useful feature when you need to quickly start a development webserver.
ntc devenv-01 ~/lab/webserver $ python3 -m http.server
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
Step 4¶
In your web browser of choice, open the following website: http://YOUR_VM_IP:8000 replacing your VM's IP as appropriate. You should now see a "Hello there!" message on the page.
Step 5¶
The webserver was started by using a module that ships with python. You will now create a separate python script that works as a web client, using a third-party library called requests.
Leave the webserver running in the current terminal and open a new one. You may do this using the Terminal -> New Terminal menu or by clicking the + sign in the top right section of the terminal panel.
Step 6¶
Create a new folder called webclient inside of /home/ntc/lab, then change into it.
ntc devenv-01 ~/lab $ mkdir webclient
ntc devenv-01 ~/lab $ cd webclient/
ntc devenv-01 ~/lab/webclient $
Step 7¶
Since you want to keep your lab VM "clean", you decide to install the requests library inside of a python virtual environment. This also allows for easier future packaging of your application.
Create a requirements.txt file containing the word requests. This will tell pip what to install. Verify its contents and location as per the output below.
Step 8¶
In the webclient folder, create a new python virtual environment called venv and activate it. Confirm that the python interpreter is running out of this new environment.
ntc devenv-01 ~/lab/webclient $ python3 -m venv venv
ntc devenv-01 ~/lab/webclient $ source venv/bin/activate
(venv) ntc devenv-01 ~/lab/webclient $ which python
/home/ntc/lab/webclient/venv/bin/python
Step 9¶
Check what additional python packages are installed using pip list, then install your project's dependencies using pip install -r requirements.txt
(venv) ntc devenv-01 ~/lab/webclient $ pip list
Package Version
------------- -------
pip 20.0.2
pkg-resources 0.0.0
setuptools 44.0.0
(venv) ntc devenv-01 ~/lab/webclient $ pip install -r requirements.txt
Collecting requests
Downloading requests-2.24.0-py2.py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB 349 kB/s
Collecting chardet<4,>=3.0.2
Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
|████████████████████████████████| 133 kB 27.9 MB/s
Collecting certifi>=2017.4.17
Downloading certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
|████████████████████████████████| 156 kB 46.2 MB/s
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Downloading urllib3-1.25.10-py2.py3-none-any.whl (127 kB)
|████████████████████████████████| 127 kB 54.9 MB/s
Collecting idna<3,>=2.5
Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
|████████████████████████████████| 58 kB 8.2 MB/s
Installing collected packages: chardet, certifi, urllib3, idna, requests
Successfully installed certifi-2020.6.20 chardet-3.0.4 idna-2.10 requests-2.24.0 urllib3-1.25.10
Step 10¶
Create a new file in the /home/ntc/lab/webclient folder called get.py. In it, place the following code:
Step 11¶
Making sure that you are still in the terminal that has the virtual environment active, execute the script using the python3 get.py interpreter.
You can always check whether the virtualenv is active by looking for the bash prompt (venv) prepended name or by using the
which pythoncommand.
Success!
Task 2 - Automating Common Python Tasks¶
Since running the get.py web client script requires a virtual environment with dependencies installed, it's common practice to use other tools to automate the creation, activation and updates of these dependencies.
You will use the GNU Make utility to manage the virtualenv and run the application.
Step 1¶
In the /home/ntc/lab/webclient folder, create a new file named Makefile and copy/paste in the following contents:
.PHONY: clean run
default: run
run: venv
. venv/bin/activate; python3 get.py
venv: venv/bin/activate
venv/bin/activate: requirements.txt
test -d venv || python3 -m venv venv
. venv/bin/activate; pip3 install -Ur requirements.txt
touch venv/bin/activate
clean:
rm -rf venv
In VSCode, open the Command Palette (Ctrl-Shift-P) and select Convert Indentation to Tabs. Save the file.
Most code editors convert tabs to spaces by default, but Makefiles require tabs for indentation, otherwise they will not work.
Step 2¶
Review the makefile. It defines the following "targets" or actions:
- run (the default): it activates the virtualenv and executes the
get.pypython script - venv (a shorcut for venv/bin/activate): it ensures that the virtualenv is created and the dependencies specified in the
requirements.txtfile are installed - clean: it removes the virtualenv
Step 3¶
Open a fresh terminal and navigate to the /home/ntc/lab/webclient folder. Do NOT activate the virtualenv!
Step 4¶
Use the command make without any arguments. It will execute the run default target.
Step 5¶
Run the command make clean to delete the virtual environment. Confirm by using ls.
ntc devenv-01 ~/lab/webclient $ make clean
rm -rf venv
ntc devenv-01 ~/lab/webclient $ ls
Makefile get.py requirements.txt
Step 6¶
Now run make again. It will detect that the virtualenv is missing, create it, and then run the script successfully.
ntc devenv-01 ~/lab/webclient $ make
test -d venv || python3 -m venv venv
. venv/bin/activate; pip3 install -Ur requirements.txt
Collecting requests
Using cached requests-2.24.0-py2.py3-none-any.whl (61 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached urllib3-1.25.10-py2.py3-none-any.whl (127 kB)
Collecting chardet<4,>=3.0.2
Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting idna<3,>=2.5
Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
Installing collected packages: urllib3, chardet, idna, certifi, requests
Successfully installed certifi-2020.6.20 chardet-3.0.4 idna-2.10 requests-2.24.0 urllib3-1.25.10
touch venv/bin/activate
. venv/bin/activate; python3 get.py
<h1>Hello there!</h1>
Feel free to play around with the various make targets (i.e. run, clean, venv).
Task 3 - Automating Tasks Using Invoke¶
Invoke is a python task execution tool & library that offers a pythonic way of implementing make functionality.
You will use invoke to automate code formatting and linting for your webclient application. The black uncompromising python formatter and the linter pylint are already installed on the machine.
Step 1¶
In the /home/ntc/lab/webclient folder, execute the black formatter in check and diff mode. It should show you output as below if you did not modify the get.py code.
ntc devenv-01 ~/lab/webclient $ black --check --diff get.py
--- get.py 2020-08-06 20:02:29.024221 +0000
+++ get.py 2020-08-06 20:39:26.787799 +0000
@@ -1,6 +1,6 @@
import requests
-response=requests.get(url='http://localhost:8000')
+response = requests.get(url="http://localhost:8000")
print(response.text)
would reformat get.py
Oh no! 💥 💔 💥
1 file would be reformatted.
As you can see, it would add some spacing around the = sign and convert the single quotes into double quotes.
Step 2¶
In the /home/ntc/lab/webclient folder, execute pylint against the get.py file. It should show you output as below:
ntc devenv-01 ~/lab/webclient $ pylint get.py
************* Module get
get.py:3:8: C0326: Exactly one space required around assignment
response=requests.get(url='http://localhost:8000')
^ (bad-whitespace)
get.py:1:0: C0114: Missing module docstring (missing-module-docstring)
------------------------------------------------------------------
Your code has been rated at 3.33/10 (previous run: 3.33/10, +0.00)
Step 3¶
Before fixing the issues, first create the equivalent of a Makefile for invoke. This is the tasks.py file (you will need to create it). In it, you define two tasks, one that runs black and the other that runs pylint, just as you did in the previous steps.
ntc devenv-01 ~/lab/webclient $ cat tasks.py
from invoke import task
@task
def black(context):
context.run("black --check --diff get.py")
@task
def pylint(context):
context.run("pylint get.py")
Step 4¶
Now run invoke, listing the tasks it supports.
Step 5¶
Ask invoke to execute the black task.
ntc devenv-01 ~/lab/webclient $ invoke black
--- get.py 2020-08-06 20:02:29.024221 +0000
+++ get.py 2020-08-06 20:47:49.437627 +0000
@@ -1,6 +1,6 @@
import requests
-response=requests.get(url='http://localhost:8000')
+response = requests.get(url="http://localhost:8000")
print(response.text)
would reformat get.py
Oh no! 💥 💔 💥
1 file would be reformatted.
Step 6¶
Ask invoke to execute the pylint task.
ntc devenv-01 ~/lab/webclient $ invoke pylint
************* Module get
get.py:3:8: C0326: Exactly one space required around assignment
response=requests.get(url='http://localhost:8000')
^ (bad-whitespace)
get.py:1:0: C0114: Missing module docstring (missing-module-docstring)
------------------------------------------------------------------
Your code has been rated at 3.33/10 (previous run: 3.33/10, +0.00)
Step 7¶
Let's fix the code. Let black actually reformat the code, instead of reporting the changes it would make. Run the command black get.py.
Remember: if you run black without any parameters, it will reformat the files without asking any questions.
Step 8¶
Run invoke black again to check that it now succeeds.
It is now, indeed, happy. Well done!