Compare commits

...
22 Commits
Author SHA1 Message Date
jimmy 27d60400e7 The islands draft. 2024-10-08 21:15:29 -04:00
jimmy 7912684880 Add uncommited draft. 2024-05-15 21:03:09 -04:00
jimmy 19bae04eaa Add post on rigt side of brain book. 2024-05-16 01:00:53 +00:00
jimmy 89fc456ec2 Add k3s certificate draft. 2024-02-28 01:59:21 +00:00
jimmy c2253f11a5 Add scrum review article. 2024-02-24 12:07:31 +00:00
jimmy dd50547005 Start rust ros2 draft. 2024-02-17 03:22:11 +00:00
jimmy 9441a91eb5 Publish sqlx article. 2024-02-16 02:00:48 +00:00
jimmy 8a83a34eb8 Finish sqlx draft. 2024-02-16 01:51:22 +00:00
jimmy 9ae6e00ccf Write up for the sqlx db script. 2024-02-14 03:47:05 +00:00
jimmy e96145b4db Start sqlx tutorial. 2024-02-14 00:36:52 +00:00
jimmy 9a84c31885 Reddit blogging review outline. 2024-02-13 00:57:36 +00:00
jimmy 9abbf33bfc Finish rename. No need to sync to til anymore. 2024-02-11 23:24:32 -05:00
jimmy 5e0cbb0b45 Rename and reword segment on what I'm focusing on. 2024-02-12 04:11:31 +00:00
jimmy 40879444eb Typo. 2024-02-11 13:16:06 +00:00
jimmy 598061cc01 Fix typo. 2024-02-11 03:17:05 +00:00
jimmy 8dce7c8714 Publish git post. 2024-02-11 01:02:09 +00:00
jimmy 134f52b118 More flash out of git article. 2024-02-10 03:29:52 +00:00
jimmy 617bb674ff Flesh out git article. 2024-02-09 04:24:21 +00:00
jimmy eb2e1b6852 More editing. 2024-02-07 02:11:50 +00:00
jimmy 6531f3d2f0 More typos. 2024-02-07 02:08:23 +00:00
jimmy df5e9dd2a3 More typos 2024-02-07 01:49:39 +00:00
jimmy 85339c3951 Fix typo. 2024-02-07 01:44:28 +00:00
16 changed files with 778 additions and 65 deletions

No files matched your search

+1 -1
View File
@@ -4,4 +4,4 @@ render:
quarto render quarto render
sync: sync:
rsync -a _site/. packaging@vpn.jpace121.net:/var/www/til.jpace121.net/. rsync -a _site/. packaging@vpn.jpace121.net:/var/www/blog.jpace121.net/.
+26
View File
@@ -0,0 +1,26 @@
---
title: "Why Everytime I Think About Getting a Pet I get a Plant"
author: "James Pace"
date: "2024/02/27"
---
Every once in a while, particularly during the late winter when the seasonal depression
really starts to hit, I start really wanting a pet.
My thought process is fairly repeatable and can be easily summed up here as:
1. I live in a small apartment and I wouldn't feel right trapping a dog
or cat in my apartment all day, so I can't get one of those.
2. Even if I wasn't worried aboutt the size, I'm not at my apartment enough to take
care of one during the work week.
3. Somehow people take care of kids????
4. Ok, so a big animal won't work, but what about something small, like a fish or
something?
5. I research fish, and realize none of the options are interesting or would work in a
small tank.
6. Start looking at hermit crabs, realize I don't really have a good spot for a 10 gallon
tank, and then start looking elesewhere.
7. Start looking at worms or ants.
Unfortunately, I live in a small apartment and don't have the space to properly take
care of a cat or dog so I don't buy one.
+32
View File
@@ -0,0 +1,32 @@
---
title: "Review of Reddit's Advice on Blogging"
author: "James Pace"
date: "2024/01/01"
---
<!--
What: Summary of current advice on blogging from r/blogging and r/just start.
Summary of each subreddit and general conclusions.
Both sites focus on generating income from blogging by winning search traffic.
Specific advice
1. Pick a microniche to focus on.
2. Build out from the niche with clear purpose.
3. Decide on articles by looking for keywords from searhc traffic.
4. AI is important going forward.
- Mixed amounts of applications.
- A lot of concern about replacing search.
5. Google "Helpful Content Update" significantly reduced search traffic.
- Relying on google is fickle.
6. Profiting from ads requires a lot of views.
Better to sale own content.
My takeaways:
1. If Google is so fickle, better to try to find traffic from somewhere
other than Google.
- Why would someone want to come my site in once I got it in their face?
2. Not worth it to rely on Ads.
Need to use the blog traffic to build trust.
Building trust conflicts with doing bad affiliate advertising.
-->
+41
View File
@@ -0,0 +1,41 @@
---
title: "Draft"
author: "James Pace"
date: "2024/01/01"
---
Problem: k3s client certs, including the admin cert expire after 1 year.
To see cert:
```
cat ~/.kube/config | grep client-certificate-data | cut -f2 -d : | tr -d ' ' | base64 -d | openssl x509 -text -out -
```
```
ssh jimmy@192.168.1.103
sudo systemctl stop k3s
sudo k3s certificate rotate
sudo systemctl start k3s
```
```
sudo cp /etc/rancher/k3s/k3s.yaml ~/j7s-cluster.yaml
sudo chown jimmy:jimmy ~/j7s-cluster.yaml
exit
```
```
rm ~/.kube/j7s-cluster.yaml
scp jimmy@192.168.1.103:~/j7s-cluster.yaml /home/jimmy/.kube/.
# Use text editor to fix IP in file.
```
```
export KUBECONFIG=~/.kube/j7s-cluster.yaml
kubectl get nodes
```
+23
View File
@@ -0,0 +1,23 @@
---
title: "ROS2 Rust Options"
author: "James Pace"
date: "2024/01/01"
---
<!--
ros2_rust
- Most official
- Good non-ROS rust integration
- no async
r2r
- good async
- most rusty
- no need to do weird things with messages
- support for colcon is weak
- one real maintainer.
safe_drive
- Good colcon support
- Good async support
- Good in between of ROS and Rust
- Not heavily adopted.
-->
-49
View File
@@ -1,49 +0,0 @@
---
title: "Summary of Free Self Hosted Git Options"
author: "James Pace"
date: "2024/01/12"
draft: true
---
<!--
Why:
1. All sorts of programming tools support git and using git for versioning.
2. Expecially with gitops being a real thing.
3. But we want to maintain control and be able to still operate completely offline.
Options:
1. Gitea
2. gitlab
3. onedev
4. gerrit
gitea:
1. Lightweight
2. Easy to host, just add a docker container.
3. Not the most mature open source community based on their
github PRs.
4. They host their development on github...
5. Heavy development in China, which is not great for US Defense
oriented companies
gitlab:
1. Most feature complete, with really good integrations with all kinds of stuff.
2. Almost every thing I've seen integrates with gitlab.
3. Fat. Hosted on a VM with 4Gb of RAM, the server kept getting OOM killed when I
was just logged into the admin panel. I currently host all my infrastrture
apps in a single VM with 4Gb of ram.
Onedev:
1. Has lots of features.
2. LIghtweight
3. Only a single developer doing most of the development, no real open source community.
4. Lots of little bugs that added up.
gerrit:
1. Had the bet review process
2. Really easy to write custom integrations with.
3. Felt "powerful" using it.
4. Code display interface sucks.
5. Workflow very different than github.
-->
+67
View File
@@ -0,0 +1,67 @@
---
title: "The Islands: Part 1"
author: "James Pace"
date: "2024/10/08"
---
> "The Islands" is a fictional series that take place in an chain
> of islands in an alternate universe.
>
> I'm writing this as an excuse to improve my writing skills,
> but I hope you also can derive some joy from it.
>
The boats constant bobbing up and down in the rough seas made the island up
ahead appear even more intimidating than it did on a normal day.
We may have been pulling up to the southern beach of the island, but it was the
mountain that set on the northern shore that dominated the scene.
The mountain was grey and almost dreary looking, with asymmetric sections
that jutted out every such way, almost like little bolts of lightning.
The southern beach, where we were headed, was gorgeous, with the blue water
reflecting the image of the palm trees growing out of the pearl white sand.
It was hard to miss the contrast.
"Mahalo!"
I hear a deep friendly voice yell as the boat bounces into the dock.
A cheery round man waves at us passengers from dock as the captain
moors the small charter boat.
"Welcome, welcome", the man says at no one in particular.
"Make sure to grab all your belongings and form a line!", he says cheeringly,
"I'll bring you through customs shortly."
The boat is quickly emptied and we follow the man through a gate towards the custom
building.
The man starts chatting with the man at the front of the line and provides some tips
for the best places to eat on the island.
The man is extremely cheerful.
I notice the irony that we're following short Santa Claus through a gate with guards
holding guns, fingers on the trigger, on each side.
The customs building is built from rock from the mountain and looks like a fortress.
If it wasn't for the men in the black uniforms holding M16s, you could be forgiven
for assuming you were in a medieval fortress.
On the left side of the door into the customs office is a sign that reads "Welcome!".
On the right is a shield for the Second Royal Regiment, which proudly displays the motto
"As the moutain protects the north we protect the south." under a drawing of a body crushed
under a boulder, only the legs sticking out.
I found my self mulling on the contrast of a "Welcome" sign next to a sign showing death,
when out guide smiled at me and said "Your next!" while pointing towards a desk where a
custom's agent stood.
I borught my brain back into reality a smiled at the guide before walking towards the agent.
I already had all the paperwork I needed in hand.
The customs agent looked through all my paperwork and then looked again, before looking at me
in the eye and completely dead pan saying "You may proceed through the double doors.
Enjoy your stay."
I walked through the heavy wood doors, and was immediately attacked by the light.
I took a deep breath in and then repacked my documents.
It was time to make it to my next destination on the island.
+1 -1
View File
@@ -2,7 +2,7 @@ project:
type: website type: website
website: website:
title: "TIL" title: "Blog"
search: false search: false
navbar: navbar:
left: left:
+5 -6
View File
@@ -9,14 +9,13 @@ listing:
sort: "date desc" sort: "date desc"
--- ---
*Things I've Learned* or TIL is a site focused on things I've learned Welcome to my blog, dedicated to things I've learned and am currently learning.
and found interesting at some point in my life.[^1]
I'm currently writing a series of fiction called "The Islands" as an excuse to
draw.
A lot of the "things" will be centered around robotics, programming, and
open source software.
[^1]: I know TIL normally stands for Today I Learned, but just focusing on what I learned
today isn't nearly as interesting.
## Latest Posts ## Latest Posts
+5 -6
View File
@@ -43,19 +43,18 @@ will:
This may seem confusing, but if you've worked with Fusion360 or Creo, you've basically done This may seem confusing, but if you've worked with Fusion360 or Creo, you've basically done
the same thing (see the timeline in the bottom of the window in Fusion360 or the history in Creo). the same thing (see the timeline in the bottom of the window in Fusion360 or the history in Creo).
I've found that it once the idea clicks what's going on isn't that hard to work out. I've found that once the idea clicks what's going on isn't that hard to work out.
What is a pain to work out is when something goes wrong. What is a pain to work out is when something goes wrong.
Its very hard to introspect what's happening at each step. It is very hard to introspect what's happening at each step.
Particularly when so much is wirtten on basically one line, which isn't a requirment per se, Particularly when so much is written on basically one line, which isn't a requirment per se,
but definitely the suggested workflow. but definitely the suggested workflow.
Regardless, the actions in each of the lines is extremely dependent on state hidden in the workplane Regardless, the actions in each of the lines is extremely dependent on state hidden in the workplane
that is basically impossible to tease out. that is basically impossible to tease out.
What I've found more challenging is it is really hard to organize things in a way that allows for code It is also really hard to organize things in a way that allows for code reuse.
reuse.
Specifically, I've worked on two projects where I've wanted to reuse sketches in multiple places, Specifically, I've worked on two projects where I've wanted to reuse sketches in multiple places,
and in neither case could I really find a nice way to do that while using the Fluid API. and in neither case could I really find a nice way to do that while using the Fluent API.
I've found the Direct API to make this much easier. I've found the Direct API to make this much easier.
The Direct API is much closer to "normal" OOP programming, where objects are made by calling The Direct API is much closer to "normal" OOP programming, where objects are made by calling
+1 -1
View File
@@ -1,5 +1,5 @@
--- ---
title: "How I Deploy Static Sites With Nginx." title: "How I Deploy Static Sites With Nginx"
author: "James Pace" author: "James Pace"
date: "2024/01/14" date: "2024/01/14"
--- ---
+1 -1
View File
@@ -42,7 +42,7 @@ of `libm::powf`.
When compiling with the default optimization level (`-O3`), I saw a similar performance When compiling with the default optimization level (`-O3`), I saw a similar performance
increase, but not when I switched to compiling with `--release`, which I found interesting, increase, but not when I switched to compiling with `--release`, which I found interesting,
and which underscores the importance of building in `--release` which shipping Rust binaries. and which underscores the importance of building in `--release` when shipping Rust binaries.
[libm]: https://github.com/rust-lang/libm [libm]: https://github.com/rust-lang/libm
[micromath]: https://github.com/tarcieri/micromath [micromath]: https://github.com/tarcieri/micromath
+77
View File
@@ -0,0 +1,77 @@
---
title: "\"Drawing on the Right Side of the Brain\" Review"
author: "James Pace"
date: "2024/05/15"
---
A few weeks ago I visited the National Portrait Museum during a business trip to
DC and was inspired by the paintings at the museum.
While I've historically not been a huge fan of art museums, looking at the
portraits and reading about the individuals in the portraits and what they
accomplished or went through in life was inspiring, and after coming back
home, I felt inspired to learn to make portraits myself.
Particularly, I was interested in drawing with a pencil.
The minimal amount of equipment required to generate something cool
compared to other art forms interested me.
I've never been an artist and wouldn't consider myself a good drawer
even when I was a little kid, so learning to draw was going to be a tall
order.
I did some research online on good books on drawing, and decided to pick up
two to get started.
The first one of those two that I read was "Drawing on the Right Side of the
Brain" by Betty Edwards.
The book's main thesis is that most thinking in modern life is dominated
by the left side of the brain, while good drawing is dominated by the right
side.
Thus, to become good at drawing, we need to learn to silence the left side
of our brains and let the right side take over.
Good drawing can further be split between five basic skills:
1. the perception of edges,
2. the perception of spaces,
3. the perception of relationships,
4. the perception of lights and shadows, and
5. the perception fo the "gestalt"
which, again, are all done best by the right side of the brain.
The book contains a number of exercises all aimed at exercising that
side and each aimed at improving one of the five basic skills.
I found the book did a good job of teaching the first three skills.
After the book, which I finished in approximately a week, I was much
better and more comfortable at generating contour drawings than I was
at the beginning.
For example, the weekend I finished the book, I drew the outline of an
empty Starbucks cup while sitting at Starbucks and generated something
that looked good, in my opinion, which I would not have been able to
do before I read the book.
I was also able to look at and the copy a cartoon of Snoopy into my sketchbook,
which would have been a struggle for me before I started the book.
(I said I was bad at drawing.)
The focus on "drawing what you see" I think is good, and something that I can
imagine I will keep using and having to remind myself to practice long term.
I don't believe the book did a good job teaching the perception of lights and
shadows.
After I read the book, I had no confidence in my ability to shade a drawing.
The book's focus is also very heavy on drawing what you see, and did nothing
to teach drawing from your imagination.
While I can't say my skills got worse, I'm not sure how to apply most
of the tools from the book when drawing from my imagination.
I found the book occassionally got a little too pseudo-scientific for my liking.
Particularly, there was a number of pages explaining how every kid goes through some
set of drawing growth, which didn't match my experience as a kid that really never drew
at all.
Further, the book spends a lot of time talking about brain sideness and going into
the "science" of brain sideness, while proof is building up that the reality of
how the brain works is not nearly that simple.
With that being said, the examples provided by the book were still seemingly beneficial.
In conclusion, I think the book was a good purchase, both worth the time I put into it
and the money I spent on it.
While not perfect nor the end all be all, I do think this book is an excellent starting
point in learning how to draw, and would recommend any one else starting down that
journey to check the book out.
+94
View File
@@ -0,0 +1,94 @@
---
title: "Scalable Scrum Frameworks"
author: "James Pace"
date: "2024/02/24"
---
Scrum is a popular lightweight framework that defines how a single team can
work towards a goal and is very popular in software engineering circles.
Scrum is the most prescriptive of the frameworks that claim to be
"Agile", and is probably the initial approach most teams reach to
when they are trying to be more "Agile".
If you've worked on a "Sprint" to work down tasks in a "Backlog", you've
probably been doing Scrum.
The official definition of Scrum is found in the
[Scrum Guide](https://scrumguides.org/index.html).
The big idea in Scrum is there is a single "Product Owner" who is accountable
for managing a "Product Backlog", which is a list of tasks that need to be
done for a project.
Teams work to complete those tasks by completing a "Sprint" over a fixed time
interval.
A "Sprint" involves:
1. Defining a goal for the end of the sprint.
2. Picking which tasks from the "Product Backlog" will be worked on
for the sprint.
3. Working on those tasks, and communicating daily to make sure there are
no blockers.
4. At the end of each sprint, delivering some sort of deliverable working
towards the final deliverable (called an "increment"[^increment]).
5. Doing a retrospective and adjusting the process and backlog given
any information learned since the last sprint.
The Scrum Guide largely defines how a **single** Scrum team is built and works.
There are three popular approaches to extend Scrum to work for **multiple** teams:
1. Nexus
2. LeSS
3. SaFE
## Nexus
[Nexus](https://www.scrum.org/resources/online-nexus-guide) is the most light weight
and least prescriptive of the three aforementioned options.
Nexus starts with multiple scrum teams, all working on one product, who practice Scrum as normal
working from a single backlog of tasks.
To keep the individual Scrum teams from stepping all over each other, Nexus adds a "meta" Scrum team
called the "Nexus Integration Team" that is responsible for the coordination between teams, and
for combining the output of the inner Scrum teams into a single combined output.
The integration team is made up of selected members of the inner Scrum teams.
Of the three options in this article, I would lean towards Nexus if I was running a small start up
that had just outgrown every one being in one Scrum team.
Nexus would be easiest to apply, with the least amount of extra work and management layers.
## LeSS
[LeSS](https://less.works/) (for LargE Scale Scrum) is similar to Nexus, in that if you squint, it is
basically inner Scrums wrapped by a bigger Scrum, but the bigger Scrum doesn't have its own team.
Compared to Nexus, LeSS is also a lot more prescriptive.
Where Nexus leaves a lot up to the reader to figure out, LeSS provides a lot of guidance, answering
a lot questions in their guide that Nexus leaves up to the reader.
LeSS suggests roughly the following process for a sprint:
1. Sprint Planning 1, where the product owner communicates the things that need to be done
to all of the teams, who proportion the work out amongst themselves.
2. Sprint Planning 2, where the teams work to convert what needs to be done into individual
tasks that can be worked on.
3. Teams then go off and do Scrum to work on those tasks, refining the backlog and communicating
with eachother as they go.
4. At the end of the Sprint one Sprint Review is completed with all teams, and then two Retrospectives
are completed, one for each team and one for the Scrum as a whole.
Of the three options in this article, I would lean towards LeSS if I was in a small but already established
company.
I think the extra prescriptiveness of LeSS can help pre-answer questions people will have in a company that
is big enough that the person making the decision to switch can't individually talk to everyone, but also
small enough to not have decades of internal politics to wade through.
## SAFe
[SAFe](https://scaledagile.com/) is the most prescriptive of the methods in this article,
and arguably the least agile.
SAFe has multiple complicated layers of management, but all the complicated levels
are clearly documented with their explicit roles spelled out clearly.
My impression is SAFe only really makes sense in a large organization where there already
is a multilayered management system, and that system is seen as good.
[^increment]: Think "incremental deliverable" but with less letters.
+169
View File
@@ -0,0 +1,169 @@
---
title: "Summary of Free Self Hosted Git Options"
author: "James Pace"
date: "2024/02/10"
---
# Introduction
A few years ago I started setting up my own self hosted
development stack [^dev-stack-definition] at home in Kubernetes.
Particularly, I currently self host gitea, a server hosting apt and
rpm repositories, a CI stack built with Tekton, and a container registry.
This article will focus on why I chose to self host anything at all,
why I chose to self host a git forge specifically, and why I chose
Gitea from the different options on the market.
# Why self host a git forge?
Cloud hosted development stacks like Github and Gitlab are extremely popular and
are used by a number of open source projects, individuals, and companies.
Using Github removes a lot of friction from taking community contributions because
it has a well understood workflow and a huge proportion of your potential contributors
already have an account there.
I don't have statistics to back up the claim that a number of companies use the
hosted on the cloud version of both tools, other than the option to buy both exist,
and there is no shortage of articles proclaiming the awesomeness of using B2B subscription
SaaS products.
I'm not a huge fan of putting components that are integral to our ability to do work
on (and only on) the internet in the defense and "dirty" commercial sectors.
24/7 good internet connectivity is not something we should assume exists in the sectors
we want robots to operate in, and we should build our workflows with that in my mind, picking
tools that we can bring with us into the zones we operate in.
This neccessitates a very "Edge" focused mentality towards the cloud, which biases my perception
towards picking tools I can self host.
(Particularly, tools I can self host on hardware that can be brought with me when I can't assume
24/7 connectivity.)
My original goal in setting up my own development stack was to see how far I could reasonably
build my own "Cloud on the Edge" that could be brought with me as part of a disconnected
command center if the need arised.
A git forge specifically is a very important part of a modern development stack.
Git forges are the primary way code is shared between developers, and without
a git forge modern software development in teams would look radically different.
Git forges are more and more being used as the single source of truth for things like
configuration management, with modern CD workflows basically boiling down to pull
something from a git repo and do the thing in it.
# Why I chose gitea?
I currently use gitea for as my personal git forge.
Gitea is an open source git forge that heavily borrows from Github's
UI and feature set.
Someone who has used GitHub should reasonable be able to "guess" how to
do the primary tasks they need to do on Gitea drawing on that experience.
Gitea is extremely light weight, with a number of people online running
it on Raspberry Pi's.
A number of different organizations use Gitea in some capacity:
1. OpenStack uses it as a tool to browse code.
(They do code reviews with Gerrit.)
2. Blender hosts their main git forge on Gitea.
3. Defense Unicorns puts a Gitea instance in their disconnected K8s clusters
that are provisioned with Zarf.
There are two reasons I chose to go with gitea.
1. It is extremely light weight and easy to host.
I was able to get a test version running on my laptop in a container running very quickly.
Once I decided to fully move to it, setting it up on Kubernetes was fairly straight forward.
I've not looked at its resource usage on my currenty cluster, but I know pre-Kubernetes
I ran it on a Vm with less than 1Gb of RAM.
2. It has the best community support.
When looking at other tools that integrate with a git forge, I noticed
pretty much all of them natively have support for Github, most of them also
supported self hosted Gitlab, and if they support anything else without using
SSH directly, it is Gitea.
With that being said, I did have some concerns initially and spent a lot of energy looking
for better alternatives.
Particularly:
1. Looking at their public Github, I didn't get the feeling the project was very mature.
It's hard to put a finger on why I felt (and still feel) that, but there is something about
building something that is so unbashedly a clone that feels immature to me.
2. The maintainers are mostly in China, which makes it challenging to ever push at work.
This may come off as Xenophobic, but our customers are and sort of are
supposed to be distrustful of other countries, and those requirements flow down.
# Other Options
## Gitlab Community Edition
One of the options I really wanted to like was Gitlab, particularly their open source
free edition.
As I mentioned earlier, outside of Github, Gitlab has the best community support, and I know
of a number of Open Source projects that aren't on Github that use it.
Particularly:
1. Debian Salsa is a "hosted by the Debian project" Gitlab instance.
2. Red Hat uses the Cloud version for some of their projects, most interestingly
to me Centos Automotive.
Unfortuntely, Gitlab is fat, requiring an insane amount of RAM to just idle.
I tried running it on my laptop in a VM with 4Gb of RAM, and with the system idling, and
the only thing happening being me in the admin panel, browsing, the server kept getting OOM
killed.
Their docs say the minimum amount of RAM is 4Gb and they are not kidding.
## Onedev
Onedev is an all in Git forge largely produced by one guy.
I really wanted to like Onedev, and ran it as my primary Git forge before
my last install of Gitea.
Unfortunately, there were just too many little UI bugs that added up so I
couldn't really suggest anyone else use it at like work, and again its
mostly developed by one guy.
It ran great on a VM with very little RAM, though.
## Gerrit
Of the git forges I looked at, Gerrit was the only one with a unique workflow that
wasn't the Github Pull Request style.
The distinguishing component of Gerrit's workflow is that a PR will only consist
of a single commit at any one point in time, and feedback is implemented by ammending
and then replacing that one commit until the PR has been accepted.
The workflow basically boils down to:
1. Make a change you want to PR on a branch with all the changes in a single commit.
2. Push the commit to a special URL. It will become a PR.
3. Get feedback.
4. Implement the feedback and ammend it to the one commit for the PR.
5. Force push the new commit, get more feedback, and repeat until the commit
gets merged.
Gerrit has some custom tooling built on top of Git which makes this easy to do.
I actually liked the workflow and think that in a fast moving software shop it would have
a lot of benefits.
It's pretty similar to merging PRs with squash merges in Gitub style workflows, except for
the squash merge is done though the whole development process not just at merge.
I think having a commit per PR, even for things in progress, would make managing integration
branches that contain multiple in progress PRs easier to manage.
I also like Gerrit's API, and it was very easy to write custom integrations with it,
The main problem I had was with Gerrit's UI for code browsing, which is basically nonexistent,
and makes Gerrit completely unusable as a code portfolio, which is one of the things my public
repos effectively are.
The other issue is that as much as I think the workflow would be nice in a fast moving software shop,
my personal projects where I don't even really do reviews just isn't that.
# Conclusion
When looking to host a git forge I looked at four different options before ultimately going with Gitea,
due to it's ease of host and good community support.
The other options I looked at each have interesting properties, but weren't suitable for what I'm doing
right now.
[^dev-stack-definition]: I'm going to use *development stack* in this article to refer to the combination
of a place to host git repos, do code reviews, run CI/CD, and host packages, for lack of a better
term.
+235
View File
@@ -0,0 +1,235 @@
---
title: "Rust: sqlx without macros"
author: "James Pace"
date: "2024/02/15"
---
[`sqlx`](https://github.com/launchbadge/sqlx) is a Rust crate that allows
for interacting with a SQL database.
One of the more popular features of `sqlx` is that it does compile time checking
of database queries (giving you some ORM-ish properties) while still allowing
you to write straight SQL queries without the abstraction of an ORM though use
of their [`query!`](https://docs.rs/sqlx/0.5.5/sqlx/macro.query.html) macro.
To make this work, `sqlx` requires:
1. access to a database at compile time, OR
2. config files to be generated from a running database everytime
the database queries are modified.
I don't particularly like either of those options, though I admittedly
have not tried either of them long term.
Requiring a running database at compile time would seem to make compilation
a lot slower.
It also adds an unexpected step (run a container hosting a development database)
before you can compile, which just feels wrong.
The configuration files are better in the sense they don't require a running
database at compile time, but it's an extra thing that has to be kept in sync
with the code.
Luckily, `sqlx` provides functions that can be used to do queries that aren't
checked at compile time for people with the same concerns as I have.
Because this isn't the recommended path though, the number of examples of
using the functions online is a little lacking, which I'll make an effort
to improve in this post.
The other thing I'll show in this post is working with JSON in Postgres.
Postgres has native support for working with JSON documents, providing
a lot of the benefits of NoSQL databases in a SQL database.
`sqlx`'s API for json documents integrates well with serde, and is reasonably
easy to work with.
# Pre-reqs
Running the example requires a running Postgres database, which I'm going to run
locally via [`podman`](https://podman.io/) using the bash script below.
```bash
#!/usr/bin/env bash
DATA_DIR="$PWD/data"
if [ ! -d "$DATA_DIR" ]; then
echo "Making data directory."
mkdir $DATA_DIR
fi
podman run --rm --name test-db --network=host \
-e POSTGRES_USER=postgres \
-e POSTGRES_PASSWORD=devpassword \
-e PGDATA=/var/lib/postgresql/data/pgdata \
-v $DATA_DIR:/var/lib/postgresql/data \
docker.io/library/postgres:latest
```
The script:
1. Defines `DATA_DIR` which will be a local host directory we will mount
in the database container and tell the database to save its files to
so they persist beyond container restarts.
2. Makes that directory if it is missing.
3. Starts a container running the official postgres image.
Environment variables are used to set some settings to values suitable
for local testing.
# Example
For the rest of this post, I will walk through an example (which can be found
[here](https://git.jpace121.net/public/sql-fun/src/branch/main/src/main.rs)).
``` rust
use serde::{Deserialize, Serialize};
use sqlx::{types::Json, Row};
```
We start by "use"-ing some structs and traits we're going to
want later, starting with the standard `serde` ones.
"Use"-ing `sqlx::Row` is important because it brings the traits we
need to work with the results of doing the sqlx queries into scope.
```rust
#[derive(Serialize, Deserialize, Debug)]
struct Doc {
age: u32,
fav_color: String,
fav_movie: String,
}
```
We next define a `Doc` struct that we will work with like JSON
in the database.
Starting with our main function...
```rust
#[tokio::main]
async fn main() -> anyhow::Result<()> {
// Connect to the database.
let pool =
sqlx::postgres::PgPool::connect("postgresql://postgres:devpassword@localhost/postgres")
.await?;
```
The first thing we do in main is to connect to the database by making a connection pool.
The `connect` function takes a string argument to control what database by what method
we want to connect to.
Specifically, in the call above, we're going to connect as the user `postgres` using the
password `devpassword` to the server running on `localhost` with the default port to the
database `postgres`.
```rust
// Make the table.
let _ = sqlx::query(
"CREATE TABLE IF NOT EXISTS test (id SERIAL PRIMARY KEY,
name TEXT,
doc JSONB
);",
)
.execute(&pool)
.await?;
```
The first thing we're going to do is generate a table for us
to work with in the rest of example.
This is the first time we've seen the `sqlx::query` function,
so we"ll pause here to talk about it.
The input to the query function is a string with, well, a SQL query
in it.
`query` returns a `Query` object which has member functions we call
to actually do the database call.
The two functions we'll use in this example are `execute` and `fetch_one`.
Both functions take the `pool` as an input and return a future with a result
that we need to await.
What's in the non-Err part of the results depends on which function we call.
For `execute` it is `()`, and thus we call `execute` when we're not expecting to
be given anything by the database.
For `fetch_one` it is a `sqlx::Row`, which contains members we can call to
query the output of our database query.
```rust
// Insert some entries into db.
// Add James.
let james_doc = Doc {
age: 30,
fav_color: "blue".to_owned(),
fav_movie: "Iron Man".to_owned(),
};
let james_doc_as_str = serde_json::to_string(&james_doc)?;
let james_add_query = format!(
r#"INSERT INTO test (name, doc) VALUES('James', '{}');"#,
james_doc_as_str
);
let _ = sqlx::query(&james_add_query).execute(&pool).await?;
```
The next thing we do is add rows to table so we have something to query.
We start by instantiating an instance of our `Doc` struct, and then using
`serde_json` to serialize it into a JSON string.
We generate and execute the database query as before, with the exception that
we make the string we're going to pass into `sqlx::query` in its own line.
One thing of interest is we define the string using a "raw string literal"
(the `r#` stuff).
The official documentation for the "raw string literal"s is
[here](https://doc.rust-lang.org/reference/tokens.html#raw-string-literals).
It's basically a special syntax that allows us to avoid having to escape characters
inside of the string.
The next block of text is basically the same as above, but for our second user,
named "Dave".
```rust
// Add Dave.
let dave_doc = Doc {
age: 25,
fav_color: "yellow".to_owned(),
fav_movie: "Spiderman".to_owned(),
};
let dave_doc_as_str = serde_json::to_string(&dave_doc)?;
let dave_add_query = format!(
r#"INSERT INTO test (name, doc) VALUES('Dave', '{}');"#,
dave_doc_as_str
);
let _ = sqlx::query(&dave_add_query).execute(&pool).await?;
```
We're now ready to query the database:
```bash
// Get james favorite color.
let fav_color_query = r#"SELECT doc->>'fav_color' FROM test WHERE name = 'James';"#;
let fav_color_result = sqlx::query(&fav_color_query).fetch_one(&pool).await?;
let james_fav_color = fav_color_result.try_get::<String, usize>(0)?;
println!("James' Fav Color: {}", james_fav_color);
```
Our first query is to find James's favorite color, which is saved in the `Doc` in
James' row.
There are three interesting new things in this block of code:
1. In the SQL query, we're going to use the special syntax `->>` to refer
to elements inside the JSON object we added to the database.
2. Since we're trying to get one element from the database, instead of
calling `execute` on the query, we use `fetch_one`.
3. The result of `fetch_one` is a `sqlx::Row`.
To get actual data out of the row we call `try_get`, which takes one
parameter, the index of the element of the row to get an element of.
There are two type parameters for `try_get`, the first the type of the
thing we're getting from the row and the second the type of the index.
```rust
// Get name based on movie.
let spiderman_query = r#"SELECT name FROM test WHERE doc->>'fav_movie' = 'Spiderman';"#;
let spiderman_result = sqlx::query(&spiderman_query).fetch_one(&pool).await?;
let spiderman_name = spiderman_result.try_get::<String, usize>(0)?;
println!("{} likes Spiderman.", spiderman_name);
```
Our second example is similar to the first, but we use the `->>` syntax to pull
something out of the JSON doc in the `WHERE` condition instead of in what we `SELECT`.
```rust
// Get james's doc.
let james_query = r#"SELECT doc FROM test where name = 'James'"#;
let james_result = sqlx::query(&james_query).fetch_one(&pool).await?;
let james_doc = james_result.try_get::<Json<Doc>, usize>(0)?.0;
println!("James Doc is: {:?}", james_doc);
Ok(())
}
```
In our final example, we pull out the whole `Doc` for James, using
the `Json` type from `sqlx` to convert between the JSON data in the
database and our `Doc` struct.