11/30/2021

Py4e

Py4e 5.2 Assignment - Solved, Working solution Only review after you have tried everything - py4e52.py. Python for Everybody is an introduction to the basics of coding in Python 3 with an emphasis on practical usage. It is intended as a foundation for students who are looking to apply Python within other academic subjects as well as in preparation for the serious study of computer science.

With this section, we transition from using Bash to learning Python. This page covers some preliminary information and getting started in Python.
Edit me
Note: Page numbers and headings referenced in the Python for Everybody reading notes are for the 6/5/2016 PDF version of the tex. This is the latest available the time the course was taught in Summer A 2020. Page numbers may differ from print or other versions.

Chuck Severance, the author of Python for Everybody: Exploring Data Using Python 3 states in his preface:

Few of my students were planning to be professionalcomputer programmers. Instead, they planned to be librarians, managers, lawyers, biologists, economists, etc., who happened to want to skillfully use technology in their chosen field.

Py4e

As far as I am aware, this also applies to most of my students, which is part of why I like this text.

Further, in Chapter 1, Chuck states:

This book assumes that everyone needs to know how to program, and that once you know how to program you will figure out what you want to do with your newfound skills.

I’m sold! I hope you enjoy this text as much as I do!

Some General Python Notes

Python 2 vs Python 3

Python 2 was released in 2000 and quickly became extremely popular. Python 3 was released in 2008, with a few changes that broke a fair bit of Python 2 code. For example python3 requires parentheses around print statements (generally equivalent to the bash echo command)

Because of these changes, many people have resisted switching to Python 3. There is still a lot of code that is Python 2 and still people writing in Python 2 (though hopefully not anymore!!).

Some of the new features of Python 3 we so good, Python 2 holdouts even made a “future” module and they do things like:

from __future__ import print_function

Py4e 9.4

But the world moves on and it is a decade later! Python 2 was initially set to “end of life” in 2015, but it wasn’t until January 1, 2020 that support was officially ended and what is supposed to be THE FINAL version of Python 2, 2.7.18, was released April 20, 2020.

But you do need to know that:

  • we’ll use Python 3,
  • pay attention when Googling for help and
  • you may run into Python 2 code, learning to recognize it (print statements are the easiest way) can help you troubleshoot incompatibilities in code you get from other sources.
Tip: Since we are talking about getting code from other sources…
Know that a coders rarely start from a blank file. Feel free to copy code from the internet (cite the source), re-use your previous code, cobble together bits and pieces from various sources, etc. I rarely write large blocks of code without taking some portions from other scripts. Do not feel that you need to start from scratch!

Python on HiPerGator

On HiPerGator, we install and maintain several version of Python for users. There is a version that comes with the operating system, and that is what is used unless you load a module for the version you want. In general, even if you don’t care too much about the version, load a python module, because these are where most user-requested Python modules are added. As an example:

Py4e

Py4e Book

The Shebang line (#!) in scripts

Remember the shebang line for bash scripts?

We need the same thing for Python scripts. Many sources on the internet will tell you to do:

But that means the script will always be run with /usr/bin/python even if that isn’t the Python you want or Python isn’t installed there.

Instead, use:

/usr/bin/env is a program to look in the environment path and check for a command, python in this case. This is more portable and allows Python to be installed wherever in the user’s path.

Ways to use Python

Maybe even more so than Bash, there are a number of ways to use Python. With Bash, we started with running commands on the command line on HiPerGator and built up to writing scripts. In addition to an interactive python prompt and scripts, Python adds Jupyter notebooks and some really nice features in VSCode and other development environments. Which of these you choose to use is largely up to you, your preferences, and the task at hand.

Interactive Python Prompt

Python has an interactive command prompt, similar to the Bash command prompt:

And you can start working in Python there. The Py4E text does a lot at the command prompt. Personally, I don’t find that terribly useful. Using the Python prompt is quite limiting and generally more frustrating than anything else. I rarely use the python prompt myself.

Python Scripts

As with Bash we can put Python commands in a text file and execute the script:

This is what I do most of the time, but requires going back and forth between editing a file and running that file. It does have the advantages of being fairly easy to run on the cluster and requiring little setup.

Editing scripts in VSCode

Similar to Bash scripts, Python scripts are plain text files. Any text editor can be used to write Python scripts. But VSCode (and others) have a number of added benefits that are worth checking out. We’ve already looked at contextual coloring, which VSCode does for many languages, including Bash and Python. This is clearly easier to read and catch errors than plain text.

Good text editors also have code suggestions (some based on machine learning algorithms that can often suggest whole blocks of code for you!) and function information popups that help you write your code. They have linters to check your syntax and flag errors. And they are configurable for how you want to code.

VSCode can also run your code and help you debug by showing you the values of variables as your code executes step by step. You can see the output in the console.

It does take some extra steps (installing the Remote Development extension from Microsoft and connecting to HiPerGator as a remote ssh server), but you can even edit and run code on HiPerGator from your local computer!

This is my main tool for development. Whether I develop locally on my own computer or remotely on HiPerGator, for larger projects, I use scripts in VSCode.

Jupyter Notebooks

Tip: For the rest of the semester, I’d generally recommend Jupyter notebooks for most things. This is the easiest way to test things and work on smaller projects.
Py4e

Jupyter is web application that creates documents that have live code, visualizations and descriptive text formatted with markdown (used in most of these pages).

Py4e Book

Jupyter can be run on HiPerGator by going to jhub.rc.ufl.edu (You do need to be on the UF network)

Note: After this page, most of my Python notes will be in Jupyter notebooks, not Markdown pages as we have seen so far. I will run those notebooks in JupyterHub on the cluster. You can do the same (make a clone of the repository), or use mybinder.org to view them.
Also note that to make serving notebooks via Binder work better, the Jupyter pages for the course are in their own repository: https://github.com/comptoolsres/Jupyter_content.

Ch 1: Why should you learn to write programs?

  • p. 3: 1.2 Computer hardware architecture: This is a nice review things we covered in the Computers lecture.

  • p. 4: Understanding programming: Like in the Bash section where we talked about top-down design and thinking out describing the steps to going to the grocery store, here the author makes the analogy to telling a story.

    Similarly, where the author talks about needing to learn the vocabulary and grammar of a language, we will find that you already know some of the grammar fom Bash. Programming languages use similar grammars and vocabularies.

  • p. 6: 1.5 Conversing with Python: We will be using Python on HiPerGator. To get to the Python command prompt, login, load the python module and type python:

  • p. 8: 1.6 Terminology: Interpreter and compiler: We briefly talked about compiling applications in the Linux section. This section goes more into interpreted vs compiled programming languages.

  • p. 10: 1.7 What is a program: Similar to the Bash scripts we wrote, Python can be written as a script.

Note: p. 11: 1.8 What is a program?
There is a copy of all the example code used in the Py4E text located at /blue/bsc4452/share/Class_Files/Py4E_files/code3/.
  • p. 12: 1.10 What could possibly go wrong? A lot! As I’ve said many times, expect errors, expect things to not work. Learning to read and understand Python error messages is a skill! Note that of the three types of errors, Python will only tell you about the Syntax errors. Logic and Semantic errors are harder to find and result in your code not doing what you wanted it to do.

  • p. 14: 1.11 Debugging: In addition to the methods here, we will look in class at using the VSCode debugger as one way to debug code as well as writing test cases to see that you get the expected answer.

  • p. 16: 1.14 Exercises: I usually do not assign the exercises, but do encourage you to think about and try some of them. They area good way to test yourself and build more fluency with the content of the chapters.

  • 1. Why Program?
  • 2. Variables, Expressions, and Statements
  • 3. Debugging Interlude 1
    • 3.4. Know Your Error Messages
  • 4. Conditional Execution
    • 4.14. POGIL Activity - Conditionals and Logic
  • 5. Functions
    • 5.17. Group Work: Functions
    • 5.18. Group Work: Functions and Strings
    • 5.19. Group Work: Functions and Conditionals
    • 5.20. Group Work: Functions and Lists
    • 5.21. Group Work: Functions with Loops
    • 5.22. Group Work: Functions with Tuples and Dictionaries
  • 6. Loops and Iterations
    • 6.6. Loop patterns
    • 6.12. POGIL Activity - Conditionals and Logic
  • 7. Strings
    • 7.17. POGIL Activity - Strings
  • 8. Files
    • 8.14. Group Work: Reading from Files
  • 9. Lists
  • 10. Dictionaries
  • 11. Tuples
  • 12. Regular Expressions
    • 12.13. Group Work: Regular Expressions (Regex)
  • 13. Network Programming
    • 13.10. HTML
    • 13.11. BeautifulSoup with Requests
  • 14. Learn Web Scraping!
    • 14.3. Two scraping examples
    • 14.4. Plans 2-3: Get the webpage
    • 14.5. Plans 4-5: Extract info from the webpage
    • 14.6. Plans 9-10: Do something with the info
    • 14.7. Putting it together
    • 14.8. Download the plans
  • 15. Using Web Services
  • 16. Object-Oriented Programming
    • 16.16. Group Activity - Classes, constructors, attributes, and methods
    • 16.17. Group Activity - Working with Multiple Classes
  • 17. Turtles: An Introduction to Objects and Classes
    • 17.1. History of Turtles
    • 17.10. Summary
  • 18. Picture Class and Methods
  • 19. Classes and Objects - the Basics
  • 20. Inheritance
    • 20.7. Case Study: Structured Postal Addresses
    • 20.8. Unit Tests
  • 21. Databases
    • 21.8. Programming with multiple tables
  • 22. Data Visualization