You may or may not have noticed that the website has changed. I built the old site,
*samkwort.com*, on a whim back in June/July 2020. I had pretty limited experience
with web development and wanted to get some familiarity with technologies that I
deemed to be quite accessible at the time. That old site was built using *Pelican*,
a static site generator implemented in Python. I remember thinking at the time that
I wanted to build a website without using any JavaScript. I was successful, though
I'm not sure why I thought that. Fast-forward three years and that old site started
to look pretty dated. Beyond that, I couldn't remember, nor did I document, how to
add new posts to the website. I figured I might as well rebuild it using modern web
technologies and take a deeper dive into the world of web development through
JavaScript.

## Goals
I wanted my new site to look better and be easier to create content for. I also
wanted to learn a bit about JavaScript. My experience had been limited to writing
a very small web scraper + JSON data server for an ESP32 project I've been working
on recently. I also wanted to learn a bit more about React; I was introduced to
React during an internship during the summer break--we were tasked with building
a custom front-end for generating reports based on Jenkins CI runs. I didn't
touch React, doing most of the backend Python-based data scraping. React seemed
pretty cool at that time. I actually sat across from a senior dev who happened
to be working on a front-end rewrite of an old Ember website. He kept saying that
any new JavaScript projects should really be written in TypeScript. I added that
to the list of things I wanted to learn.


## Implementation
I did some research into modern web frameworks and settled on *Next.js*; its
page-based routing and templating feature seemed quite intriguing, as well as
its relationship with React. It also seemed to be a bit of an industry standard.

### Layout
I started with the basic *create-next-app* template. I built the basic home-page,
creating custom *Header* and *Footer* components. Then, I created a *Layout*
component that would serve as the base for all my pages. Along the way, I picked
up a little bit of *Tailwind CSS*--styling my pages this way felt really simple.

The *Layout* element code looks like this:
```typescript
interface LayoutProps {
  children: React.ReactNode;
}

export default function Layout({ children }: LayoutProps): JSX.Element {
  return (
    <div className="mx-4 md:mx-auto md:max-w-6xl flex min-h-screen flex-col items-left p-4 md:p-24 font-mono">
      <Header/>
      <main>{children}</main>
      <Footer />
    </div>
  );
}
```
I really like the way React feels like HTML; the idea of custom components is
quite neat.

### PostList
I wanted the home-page to have a list of the most recent posts. To do this, I needed
to find a tool for parsing Markdown documents. After some research, I found the
*gray-matter* package. This parsed the Markdown into a JSON object containing data
and content fields; the data fields were generated using the YAML frontmatter of my
Markdown documents. 

I placed all my posts in a *posts* folder and wrote some code to generate a list of
Gray Matter objects. These were then sorted by the date field. I used
*getStaticProps* to ensure that the static *blogPosts* data was loaded before the
page was rendered.

Here's an excerpt of the code:

```typescript
interface HomeProps {
  postsData: Post[];
}

export async function getStaticProps(): Promise<{ props: HomeProps }> {
  const postsData = await getPostData();

  return {
    props: {
      postsData
    }
  };
}

export default function Home({ postsData }: HomeProps): JSX.Element {
  return (
    <Layout>
      <section>
        <h2 className="text-lg">Latest Posts</h2>
        <PostList postsData={postsData.slice(0, 5)} />
      </section>
    </Layout>
  );
}
```

### Dynamic routing with \[post\].tsx
*Next* offers dynamic routing, which makes it great for generating generic blog
post pages for any given Markdown document. Setting this up is fairly easy; the
[Next documentation][1] gives a pretty good example.

Once I had dynamic routes working and returning a basic hello-world type message,
I started working on parsing the Markdown blog posts into an abstracted
*BlogPost* component.

### Markdown formatting
The essence of the *BlogPost* component is Markdown to HTML conversion. I found
the perfect React component for this, [React Markdown][2]. This component takes
Markdown formatted text as its child and converts it to HTML.

I also wanted the option to display code and math in my blog posts. To set this
up, I followed the examples on the *React Markdown* GitHub page. The code
highlighting used the Syntax Highlighter package, while the math formatting
used remarkMath and rehypeKatex.

The final *BlogPost* component looks like this:

```typescript
<ReactMarkdown
  className="markdown"
  remarkPlugins={[remarkMath]}
  rehypePlugins={[rehypeKatex]}
  components={{
    code({ node, inline, className, children, ...props }) {
      const match = /language-(\w+)/.exec(className || '');
      return !inline && match ? (
        <SyntaxHighlighter
          {...props}
          style={irBlack}
          language={match[1]}
          PreTag="div"
        >
          {String(children).replace(/\n$/, '')}
        </SyntaxHighlighter>
      ) : (
        <code {...props} className={className}>
          {children}
        </code>
      );
    },
  }}
>
  {post.content}
</ReactMarkdown>
```

To style everything nicely, I used a dedicated stylesheet with Tailwind `@apply`
directives. I think it came together quite well. Also, here's some math:

$$
\begin{align*}
    D\int_\reals x^4\,u_{xx} \; dx &= -4\,D \int_\reals x^3\,u_{x} \; dx \\
                                  &= 12\,D \int_\reals x^2\,u \; dx
\end{align*}
$$


### Hosting
My domain *samkwort.com* was previously hosted by Google Domains, which was
recently sold to Squarespace. As such, I decided to migrate my domain to
Cloudflare after reading some very positive reviews online.

Like my previous site, hosting is done through GitHub Pages. I used the
[Deploy to GitHub Pages Action][4].

## Conclusion
JavaScript is weird. I definitely prefer coding in C and Python, but it was good
to branch out a little bit. I'm a big fan of the new site; making content is
super straightforward, and the design is a big upgrade.

[1]:https://nextjs.org/docs/pages/building-your-application/routing/dynamic-routes
[2]:https://github.com/remarkjs/react-markdown
[3]:https://github.com/vercel/next.js/tree/canary/examples/github-pages
[4]:https://github.com/marketplace/actions/deploy-to-github-pages







A New Website


As part of my current role at work, I'm required to automate the production of
many plots using results from PSCAD simulations. 
These plots are generated by `matplotlib`.
To streamline the plot reviewing process, we generally combine the plots into
a single PDF file. Currently, this process is incredibly slow, sometimes taking
minutes to generate a single PDF, with 50+ of pages.
As a learning exercise in Python benchmarking, and to improve performance, I
decided to undertake an investigation into this component of our simulation
workflow, hopefully removing the bottleneck.

## Current Practices
The current methodology for generating PDFs looks something like this:

1. Generate simulation data for each scenario, including plots.
2. Store folders names containing output plots in dictionary by 
   scenario number.
3. Iterate through output folders using dictionary and generate PDF
   for each scenario.

I narrowed down the performance bottleneck to step 3 above.
For this step, we have the following Python code:

```python
# Create unique pdf for each scenario
for key in pdf_groups.keys():
    pdf = FPDF(orientation='L', unit='mm', format='A4')
    
    # Loop through test in scenario, keeping track of first and last
    first_test = ""
    last_test = ""
    for test in pdf_groups[key]:
        if last_test == "":
            first_test = test

        for signal in SIGNALS_TO_CAPTURE:
            image = f"{out_folder}\\{test}\\{test}_{signal}.png"
            # Only add the graph to PDF if it exists.
            if (os.path.isfile(image)):
                pdf.add_page()
                pdf.image(image, x=0, y=20, w=300, h=150)
            else:
                print(f"File: {image} does not exist.", file=sys.stderr)

        last_test = test

pdf.output(out_folder + "\\" + first_test + "-" 
            + (re.search(r"[0-9]+",last_test)).group(0) + ".pdf", "F")
```

The code itself is $O(n\times m)$, where $n$ is the number of scenarios
and $m$ is the number of tests.

My intuition tells me that the bottleneck is somewhere in the `FPDF` package,
which is used here to embed the images in generated PDF output. We will
benchmark this entire code segment.

## Benchmarking
There are a few options for benchmarking the speed of code in Python. Some of
these are:

- [`timeit`][1]
- [`line_profiler`][2]
- [`pyperf`][3]

I'm opting to use `line_profiler` as this will give me a line by line breakdown
of my code's execution speed.

### Benchmarking Setup
To support the profiling process, we need to isolate this segment of code and
create a custom script for profiling. Here's what I ended up with.
```python
@profile
def benchmark_with_FPDF(tests, out_folder, in_folder):
    pdf = FPDF(orientation='L', unit='mm', format='A4')
    for test in tests:
        for signal in SIGNALS_TO_CAPTURE:
            image = in_folder + "\\" + test + "\\"+ test + "_" + signal + ".png"
            # Only add the graph to PDF if it exists.
            if (os.path.isfile(image)):
                pdf.add_page()
                pdf.image(image, x=0, y=20, w=300, h=150)
            else:
                print("File: " + image + " does not exist.")


    pdf.output(out_folder + "\\FPDF_benchmark.pdf", "F")
```

Note the use of the `@profile` decorator; `line_profiler` uses this to signal
which function to profile.

To profile the script with `line_profiler`, we just run:
```
kernprof -lv script_to_profile.py
```

### Results
Before using line_profiler, I ran a very crude test shown below.
```python
start_time = datetime.now()
benchmark_with_FPDF(tests, out_folder, in_folder)
total_time = datetime.now() - start_time
print(f"Benchmark took {str(total_time)}")
```
This gave me a runtime of `45.574701 seconds`. Note that the input was 24 PNG
images, generating a 24 page PDF.

The results from the intial profiling are captured below.
```python
Benchmark took 0:01:48.941374
Wrote profile results to benchmark.py.lprof
Timer unit: 1e-06 s

Total time: 108.941 s
File: .\benchmark.py
Function: benchmark_with_FPDF at line 20

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
    20                                           @profile
    21                                           def benchmark_with_FPDF(tests, out_folder, in_folder):
    22         1         59.7     59.7      0.0      pdf = FPDF(orientation='L', unit='mm', format='A4')
    23         3          1.6      0.5      0.0      for test in tests:
    24        24         25.5      1.1      0.0          for signal in SIGNALS_TO_CAPTURE:
    25        24         39.2      1.6      0.0              image = in_folder + "\\" + test + "\\"+ test + "_" + signal + ".png"
    26                                                       # Only add the graph to PDF if it exists.
    27        24       2324.0     96.8      0.0              if (os.path.isfile(image)):
    28        24        708.6     29.5      0.0                  pdf.add_page()
    29        24  108695906.0 4528996.1     99.8                  pdf.image(image, x=0, y=20, w=300, h=150)
    30                                                       else:
    31                                                           print("File: " + image + " does not exist.")
    32
    33
    34         1     242092.4 242092.4      0.2      pdf.output(out_folder + "\\FPDF_benchmark.pdf", "F")

```

According to the results, 99.8% of execution time is spent on this FPDF API
call.
```py
pdf.image(image, x=0, y=20, w=300, h=150)
```
My intuition seems to be correct; it's clear that the bottleneck is in FPDF.

## Alternatives to FPDF
After some research, I found the following tools to be candidates for replacing
FPDF.

- [`img2pdf`][4]
- [`Pillow`][5]

It seems that `img2pdf` actually uses `Pillow` internally. These `img2pdf` API
seems straightforward to use, so I'll just be testing that. 

The benchmarking function rewritten to use `img2pdf` is included below.
```python
@profile
def benchmark_with_img2pdf(tests, out_folder, in_folder):
    # Get a list of paths to the images
    paths = []
    for test in tests:
        for signal in SIGNALS_TO_CAPTURE:
            image = in_folder + "\\" + test + "\\"+ test + "_" + signal + ".png"
            # Only add the graph to PDF if it exists.
            if (os.path.isfile(image)):
                paths.append(image)
            else:
                print("File: " + image + " does not exist.")

    # Specify paper size (A4 landscape)
    a4inpt = (img2pdf.mm_to_pt(297), img2pdf.mm_to_pt(210))
    layout_fun = img2pdf.get_layout_fun(a4inpt)

    # Generate PDF
    with open(f"{out_folder}\\benchmark_img2pdf.pdf","wb") as f:
        f.write(img2pdf.convert(paths, layout_fun=layout_fun))
```

As before, we run the script again with `line_profile`.

### Results
Using my crude test script again, the img2pdf API only took 1.887416 seconds.
The `line_profiler` results confirm this:
```python
Total time: 94.2377 s
File: .\benchmark.py
Function: benchmark_with_FPDF at line 21

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
    21                                           @profile
    22                                           def benchmark_with_FPDF(tests, out_folder, in_folder):
    23         1        110.0    110.0      0.0      pdf = FPDF(orientation='L', unit='mm', format='A4')
    24         3          2.5      0.8      0.0      for test in tests:
    25        24         44.1      1.8      0.0          for signal in SIGNALS_TO_CAPTURE:
    26        24         63.1      2.6      0.0              image = in_folder + "\\" + test + "\\"+ test + "_" + signal + ".png"
    27                                                       # Only add the graph to PDF if it exists.
    28        24       3338.2    139.1      0.0              if (os.path.isfile(image)):
    29        24       1062.9     44.3      0.0                  pdf.add_page()
    30        24   93822206.6 3909258.6     99.6                  pdf.image(image, x=0, y=20, w=300, h=150)
    31                                                       else:
    32                                                           print("File: " + image + " does not exist.")
    33
    34
    35         1     410847.1 410847.1      0.4      pdf.output(out_folder + "\\FPDF_benchmark.pdf", "F")

Total time: 1.88704 s
File: .\benchmark.py
Function: benchmark_with_img2pdf at line 37

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
    37                                           @profile
    38                                           def benchmark_with_img2pdf(tests, out_folder, in_folder):
    39                                               # Get a list of paths to the images
    40         1          0.5      0.5      0.0      paths = []
    41         3          1.2      0.4      0.0      for test in tests:
    42        24          8.7      0.4      0.0          for signal in SIGNALS_TO_CAPTURE:
    43        24         25.2      1.1      0.0              image = in_folder + "\\" + test + "\\"+ test + "_" + signal + ".png"
    44                                                       # Only add the graph to PDF if it exists.
    45        24        938.6     39.1      0.0              if (os.path.isfile(image)):
    46        24         15.7      0.7      0.0                  paths.append(image)
    47                                                       else:
    48                                                           print("File: " + image + " does not exist.")
    49
    50                                               # specify paper size (A4 landscape)
    51         1          3.9      3.9      0.0      a4inpt = (img2pdf.mm_to_pt(297), img2pdf.mm_to_pt(210))
    52         1          9.8      9.8      0.0      layout_fun = img2pdf.get_layout_fun(a4inpt)
    53
    54                                               # Generate PDF
    55         1        408.9    408.9      0.0      with open(f"{out_folder}\\benchmark_img2pdf.pdf","wb") as f:
    56         1    1885623.0 1885623.0     99.9          f.write(img2pdf.convert(paths, layout_fun=layout_fun))
```

- FPDF took approximately 94.2377 seconds to complete the conversion process.
- img2pdf completed the same task in approximately 1.88704 seconds.

Therefore, by migrating our image to PDF conversion scripts to use img2pdf, we
should observe a 50x increase in PDF generations speeds. This is huge for
multi-hour batched simulations runs, which will have 50+ test cases across
multiple scenarios.

## Conclusion
There was a major bottleneck in our image to PDF conversion code. By moving
from `FPDF` to `img2pdf`, we saw a 50x increase in program execution speed.


[1]:https://docs.python.org/3/library/timeit.html
[2]:https://github.com/pyutils/line_profiler
[3]:https://github.com/psf/pyperf
[4]:https://github.com/myollie/img2pdf
[5]:https://pillow.readthedocs.io/en/stable/

Image to PDF in Python


Some recipes that I like and use fairly regularly...

### Chocolate Chip Cookies:
I've tried a lot of chocolate chip cookie recipes, and this one from Bon Appetit, [BA's Best Chocolate Chip Cookies](https://www.bonappetit.com/recipe/bas-best-chocolate-chip-cookies "BA's Best Chocolate Chip Cookies"), is my favourite.

I make the following modifications:

  - 1½ cups (200 g) all-purpose flour
      - I use high protein flour. I  prefer the texture.
  - 1 cup (200 g) (packed) dark brown sugar
      - Regular brown sugar is fine.
  - ¼ cup (50 g) granulated sugar
      - I don't put this in. It's sweet enough with the brown sugar.
  - 2 tsp. vanilla extract
      - Add more vanilla extract. It tastes good.
  - 6 oz. (170 g) bittersweet chocolate (60%–70% cacao)
      - Any dark chocolate will do, the darker the better in my opinion-- it's more rich.


### Pasta:
I have pasta a lot. It's really easy. You just need tomato paste, olive oil, and pasta-water.

You can use any vegetable/meat. I generally use bacon and whatever vegetables I have on hand. At the time of writing it's mostly asparagus/peas.

The general method is this:

  1. Cook meat/protein
  2. Cook onions/garlic/vegetables in fat/oil from meat
  3. Add tomato paste and cook until oil is infused/red. Season.
  4. Cook pasta, reserve a cup of the water.
  5. Add pasta and some pasta water to your sauce.
  6. Mix vigourously and thoroughly, adding more pasta water until a sauce forms and coats the pasta.

Don't forget to keep adding olive oil as you add more ingredients. It will take more than you think. Same with the pasta water.

These videos are good for understanding the technique:

  - [PASTA AMATRICIANA](https://www.youtube.com/watch?v=VgUo1JHM-DQ)
  - [SPAGHETTI CARBONARA](https://www.youtube.com/watch?v=90LalTBSIhE)


### Felafel:
I really like this [recipe](https://www.indianhealthyrecipes.com/falafel-recipe-chickpea-falafel "Felafel Recipe"). They've come out great every time.

I don't follow it accurately anymore, I just eyeball the ingredients and adjust it to my tastes--I generally put in more onion, garlic, cumin, and cayenne than is listed.
