Writing Web Scraped HTML to a File

I was working on a project for a client where I needed to scrape data from a Web page. I wanted to save the page to a file so that I wouldn’t be making requests to the server hosting the page each time I wanted to test my code. I was using Python3 and the Requests library. When attempting to perform the write to a file, I ran into encoding issues. This task was not as straightforward as I first imagined.
Continue reading

Statistics – Probability Density Function and Z Table

It turns out that normally distributed values are quite important in statistics. Not only because the pattern is remarkably common, the central limit theorem enables statisticians to infer conclusions about how a given treatment will affect a given population. To make such inferences, we need to learn about the Probability Density Function and a useful shortcut: the Z Table.
Continue reading

Statistics – Standard Deviation

Most people are familiar with the concepts of the mean, median, and mode. They are measures of the central tendency of a value that has measured in a given population. They tell us, in different ways, about the value of an attribute at the heart of the population, rather than at the positive or negative extremes. Other measures are used to tell us about the variation within a population. One frequently used measure of variation is standard deviation.
Continue reading

Introduction to NumPy

Lately, I’ve been studying statistics and data analysis. I have beforehand knowledge of the Python programming language, so when looking at the two most widely used programming tools applied in this domain, Pandas and R, I chose Pandas – a software library for Python. Pandas uses another library in the construction of it’s data structures (Series and DataFrames), called NumPy. This post goes over some of what can be done with NumPy.
Continue reading