Testing Functional Code

In programming, it is essential that developers can ensure that their code works as expected. One way to ensure this is unit testing. However, unit testing and many of the general testing styles are only convenient and simple for basic types of projects. For example, unit tests are great for public libraries made up of classes and functions that can easily be tested. Not only is it easy to test these, but it is simple to understand. You just make objects and call methods and check the results. However, not all code is made up in that fashion.

A clear example of this is a programming language such as my Sea language. In theory, I could take a few weeks and create proper tests for everything involved. However, creating production-level unit tests for a project of this size by yourself is tedious and time consuming, to say the least. One way I find convenient to help test my language is to simply create a sea file with code I want to test the interpretation/transpilation of and then run both the transpiler and interpreter to check the results. Another thing I did was add a debug option to print out the generated tokens, AST, and memory.

This is admittedly janky. There is no standardization for my testing process. Luckily, most of the problems I run into either generate a Python exception, or an obviously incorrect output. That said, maybe that’s only because those kinds of problems are the only ones I find and there are tons of hidden errors that are laying quietly. That overall is one of the best things about Python, and my motivation for Sea. In Python, it is really easy to write thirty lines of code and then run it and have it execute without any errors. This is because the syntax is so simple and readable that its easy to find errors before you even run the code.

Sometimes, I think its alright to not properly commit to testing code. Code that relies more on user input than parameter input (what I’ve called functional code) can be incredibly challenging to properly test. For instance, if I were to properly test Sea, I would need to create sea files that have almost every combination of valid syntax. Not only that, but I’d need combinations of valid and invalid syntax as well to make sure the code finds the errors. The point of testing is to save time and to ensure valid code. In the case of a programming language, I think its easy enough to ensure valid code and in exchange you can save almost half of the total time you’d otherwise need to be spending.

Starting off Clean

I have a noticeable tendency when I’m coding or even when I’m gaming. After a while of making progress, I often reach a point where I’ve learned enough to remake – whatever it is that I made – much better. So, I’ll get to work on modifying what I currently have in an attempt to improve it. This can work sometimes; usually, however, I end up with a lot of complexity laid out in front of me and I become lost in it. Then, I follow my tendency to simply start over.

In my Sea programming language for instance, I started without any real knowledge of how to make a language. I followed a tutorial with pretty bad code and modified it to serve my needs, learning along the way. I’ve recently added all of the combined assignment operators as well as loops. The next step is to add a method of mimicking main memory so I can add functions and proper variable scopes, as well as memory management. So, I want to start off by refactoring the code I currently have. I made decent progress but then I found myself with numerous files open without any new working code. Since I was rewriting basic features anyway, I finally gave in and started rewriting all of the Python code from scratch. The new code can currently be seen in the overhaul branch, until I finish and merge it. (If you’re reading this in the future, that link might not be useful anymore so try this compare link.)

So, I’d like to discuss the benefits and drawbacks of this habit of mine. First off, sometimes, starting off clean just helps manage the complexity of a problem. Being able to go through every file of code and determining which lines are good is incredibly useful and you can find things you otherwise won’t notice. With git and the ability to copy and paste, it takes hardly any time to reincorporate good code. However, I tend to still think of this process as something I ideally shouldn’t do. Maybe it would save me a lot of time to simply refactor what I need to. It’s possible I just need to get better at refactoring. However, it’s also possible that I’m simply new at a lot of this and in the future I’ll write more solid code from the beginning anyway.

I think, at least in this current Sea overhaul, starting off clean has been nothing but positive for me. This is code I’m very new with and having added so many features, I understand it significantly better than when I first wrote it. I think at the end of the day, both refactoring and starting off from the beginning are valid strategies. As we learn, we’ll learn which to use when. If you only need to make a few changes, then just refactor the code you already have. However, when you’re faced with a massive redesign, just take the time to rewrite it. The problem isn’t so much which you choose to do, but rather when you decide to do one over the other.

Apprenticeship Pattern – Nurture Your Passion

The key focus of this pattern is founded in the all too familiar scenario where you fundamentally enjoy the subject matter you are surrounded by at your work; however, the work itself gets in the way of your passion for the subject. An example of this would be suppose you are really passionate about software design and you enjoy taking your time making a solid product using good code. While that is your passion, and rightfully so, you may find your job is more concerned about functionality and speed. They might prefer you create shoddy code in a quick amount of time that works, rather than doing the job properly. As you might imagine, if your goal was to be employed to create solid code the way you always have, this might hurt your motivation.

The proposed solution to this predicament is to search around at your work and find something that properly interests you. Involve yourself in it and put in as much time towards it as possible. Outside of work, do the same. Find and work on interesting things that satiate your passion. I would continue the prior example to demonstrate how the solution works, except that its a pretty good example for why it doesn’t. I’m sure that in the right situation, the proposed solutions are good options. In situations such as above, however, it isn’t really possible to change what you do at work to better suit your passion. If you already work in software for the company, and the software isn’t up to your standard, where in the company can you go that is up to your standard?

Many companies have a fixed structure. Often, if you attempt to either move yourself around or change the structure itself, it is easier for the company to simply let you go and find someone else. That’s one of many problems that come from jobs being rare and workers being abundant. If jobs had to compete with one another for workers, overall conditions for workers would improve. As a quick aside: in my opinion, if the government weren’t trying its hardest to support major corporations (due to being bought out by them) and got rid of regulation that solely hurts small businesses, maybe in a more free-market economy where almost anyone can start up a company jobs could be abundant and would have to compete for workers. Speaking of idealistic views that might not actually stand up in real life, a second component to the proposed solution is to work on personal projects in your free time.

This is similarly not a valid option for many situations. Often, people simply do not have the time to work a full-time job and fulfill other responsibilities. Bringing up my example once again, you can work on quality software all week long in your free time, but writing code you know is bad and that you know could be better with just a little bit more time will always drain on you. Sometimes and for some people, it can be pretty easy to work on something you don’t enjoy. It can be easy to create something worthless and bad, but it depends on the situation and, more so, the person in particular. I personally find it extremely hard to work on things I don’t care about; even if I just have to do it badly.

It can be hard to have a strong passion for something and then have other people not allow you to do it. It isn’t even a matter of them letting you do it; it would be enough for them to do nothing. But often times, they directly work to oppose what your soul wants you to do. That said, you can’t just vilify them for it. You ended up in a bad situation but in the previous example, for instance, you wouldn’t be stuck. Maybe you misunderstood what they want or maybe they misled you. But if it’s a matter of your soul, so to speak, you can always leave the job. It isn’t always that simple; we have responsibilities and bills to pay. That’s why I personally believe that if you have the opportunity, build up your savings and investments. Sacrifice some of your standard of living now so you can afford to make a change like this in the future. Do your best to prevent yourself from getting trapped in life.

Sprint 2 Retrospective

Summary

In this sprint, I mainly did the following things:

What Worked Well

This sprint, I worked a lot with Marcos on the backend. Overall, I think that we worked well together. This was the first time I spent most of my time working with someone else and it went well. We didn’t really run into any issues because we made sure to be careful with git. I now see why its recommended to add new features on a new branch. We were able to split up the work and stay in communication.

I also had a decent understanding of OpenAPI and Node during the work. It felt like I knew what I was doing in a sense.

What Didn’t Work Well

I think the team overall just had a lot to do outside of this class. I was able to get done what I needed to during class but if I had to work on things outside of class as well, I wouldn’t have had the time and the motivation at the same time. I think that overall as a team we didn’t get as much done as I expected because people were busy with other classes.

We also still sometimes feel lost and don’t really know what questions to ask. This applies mostly to the details of the design or implementation. However, it was a lot better than Sprint 1.

What Changes Could be Made to Improve as a Team

We need to get used to typing out more info for the cards. I’ve also noticed people aren’t always moving the cards they’re working on into the “doing task” column. Besides that, I think as a team we’re doing pretty well. We work together when its necessary or convenient and manage our own work otherwise.

What Changes Could be Made to Improve as an Individual

I could dedicate more time to this class overall and I could ask teammates if they need help rather than always waiting for them to ask for it.

Apprenticeship Pattern – Unleash Your Enthusiasm

The key focus of this design pattern is the situation in which you have more enthusiasm for software development then the rest of your colleagues. Due to this, you end up holding back to fit with the group. I have a few experiences related to both sides of this. While I’ve never partaken in a software job, I have taken part in software courses. There have been plenty of times when I’m in a group project and the topic is so interesting to me I can just get to town and code almost everything. I guess rather than having it hold me back, I end up usually embracing it.

Then there are situations similar to that which I’m in now. It isn’t precisely a mirror for the apprenticeship pattern, but it’s pretty close. I find that when I have other things I need to do are when I’m most passionate about other things. For example, I’ll most intensely want to theorize about physics when I have an assignment due; however, once I actually have free time, I’m mostly content just wasting my time playing video games. Over the last month, I’ve become fixated with Sea. I had been working on creating a Minecraft Server manager in Node.js for my friends and I, then I moved onto creating a way of backing up my playlists. Then I realized it’d probably be easiest to do it in Python, so I started rewriting the code. In that process, I fell in love with Python.

I had used it before and enjoyed it, but I was almost something of an elitist. It had no strict types, it was easy to write. I treated it as if it were a beginners language. So I never really took the time to learn it. I rewrote my code in Python and learned a lot of the joys of Python such as context managers, list/set/dict/string comprehension, etc. (I just need to figure out the SQL commands and maybe I’ll eventually finish it). I had always been aware of the fact that Python running in a single threaded interpreter will never be able to perform as well as something like C. C by itself has a lot of joy to write in. Every language has its own personality you get to learn. That said, C can be tedious to write in and to read. That began my quest to create Sea – a version of the C language with Python-like syntax. I have become passionate about designing what the ideal language would be. Something modular, with high and low level features, and is easy to write and debug. Sea is just the first step in that.

I have found that I can so easily spend six hours straight coding, debugging, and refactoring Sea code. I can then go to bed and while I’m trying to fall asleep or even while I’m dreaming, I’ll be making design decisions. Classes that are otherwise fine can seem boring by comparison. Being able to just create something functional that has a clear use case feels great. At least sometimes, it can be really easy to share that enthusiasm with other students and it overall helps all of us.

Creating a Transpiler

A transpiler is a program that converts code from one programming language to another programming language. This is comparable to a compiler, which is a transpiler that converts into machine code. It is also related to an interpreter, which behaves similarly, except rather than writing new code, it performs the code.

In my work on the Sea programming language I’m making, I took a long time writing a custom system for transpiling. However, while it succeeds at managing indentation pretty well, it makes actually transpiling statements much more challenging. So, recently I’ve gone back to the drawing board and have decided to pursue the classic model. If it ain’t broke, don’t fix it.

I’m working off of David Callanan’s Interpreter Tutorial. While it’s a very useful tutorial, the code is admittedly pretty poor, as it contains a few files with hundreds of lines. I’m also using Python exceptions to carry errors, since as far as I’m aware, Python has one of the safest exception systems (unlike C++). I can safely catch and handle exceptions to create useful messages for the user. The tutorial, on the other hand, is manually passing around errors from function to function. That said, the explanations are decent and it is a very useful tutorial. I’ll just have to make a lot of modifications and refactoring after each episode in the tutorial. That said, let’s go over how a transpiler works fundamentally:

The Process

The first step in transpilation is reading the source file. The lexer goes character by character and matches them to a set of predefined tokens. These tokens define a significant part of the syntax of a language. If it doesn’t recognize symbols, it can give an error that alerts the programmer. If there aren’t any errors, the lexer will go through the entire file (or files) and create a list of these matched tokens. The order of the list encodes the order that elements appeared in the file. Empty space and otherwise meaningless syntax symbols are not passed on.

Next, the list of tokens is sent to the parser. The parser will then go through the list of tokens and create an Abstract Syntax Tree (AST). This is a tree of tokens whose structure encodes the order of operations of the language’s syntax. In this stage, the order of the list is lost; however, that order isn’t important. What matters is in what order tokens should be. For instance, the list of tokens for 5+22*3 might look something like [INT:5, PLUS, INT:22, MUL, INT:3] and the list of tokens for (5+22)*3 might look like [LPAREN, INT:5, PLUS, INT:22, RPAREN, MUL, INT:3]. The ASTs for these token lists will look something like this respectively:

Created on https://app.diagrams.net/

Lastly, you then traverse the tree using depth-first-search (DFS), or more specifically, Preorder Traversal of the tree. This means we start at the root node and we the work our way down the left side and then down the right side. This is incredibly simple to implement using recursion. Each new node you check can be treated as the root to a new tree where you can then proceed to repeat the search. This occurs until the entire tree is traversed.

In this final stage, this is also where transpilers, compilers, and interpreters differ. Until now, the same code could be used for all three. At this point, if you want a transpiler, you use the AST to write new code. If you want a compiler, you use the AST to write machine code. If you want an interpreter, you use the AST to run the code. Notice this is why there is such a performance benefit to using a compiler over an interpreter. Every time you interpret code, assuming there is no caching system in place, the interpreter has to recreate the entire token list and AST. Once you compile code, it is ready to be run again and again. The problem then comes from compiled code potentially being more complicated for higher-level language features, and thus making it a pain to write a new compiler for every CPU architecture, due to different architectures using different machine instructions.

From C to Shining Sea

The Snake

As of recently, I’ve been spending most of my personal coding time in Python. I enjoy a lot of languages and Python certainly isn’t for everything, but when you can use Python, boy is it a joy. As someone who strictly indents in any language, I love the indentation style of denoting blocks. Curly braces have their use, but the vast majority of the time, they’re purely redundant. The same goes for semicolons. I completely agree with the movement of programming languages towards spoken language. The main downfall of Python, comes from how high-level of a language it is.

Being a high-level language allows for it to be as convenient to write in as it is, however you are completely unable to use low level features. It also means Python’s performance is often much lower than that of C++ or other languages. Of course, everyone says that each language has its own use and Python isn’t meant for performance-intensive programs. But why not? Wouldn’t it be nice if there were a single modular language that had Python-like simple syntax with the features of JS, Python, C++, etc.

The Sea

Before I take on the task of creating such a language, I want to start smaller. Introducing Sea- It’s C, just written differently. I am currently working on a language called Sea which is effectively C, but with Python-like syntax. I say Python-like because much of the syntax of Python relies on internal data types. My goal is to keep Sea true to C. That is, no increase performance penalty; all of the penalty should be paid at compile time. That’s phase one. Start off with a more concise face for C. Then, I want to create libraries for Sea that take it one step further – introducing data types and functions innate to Python like range, enumerate, tuples, etc. Lastly, I want to use the knowledge I’ve gained to create the language to end all languages as described above.

I’m starting off with a Sea-to-C Transpiler, which is available on Github. In its present state, I am able to transpile a few block declarations and statements. I’m currently working on a data structure for representing and parsing statements. Once that’s made, I can add them one by one. The final result should look something like this:

include <stdio.h>
include "my_header.hea"

define ten as 10
define twelve as 12

void func():
    pass

int main():
    if ten is defined and twelve is defined as 12:
        undefine twelve
        // Why not

    c block:
        // Idk how to do this in Sea so I'll just use C
        printf("Interesting");

    do:
        char *language = "Python"

        print(f"This is an f-string like in {language}")

        for letter in language:
            pass

        break if size(language) == 1
    while true and ten == 11

    return 0

Once the transpiler is done, I want to create an actual compiler. I’ll also want to make a C-to-Sea transpiler eventually as well. I’ll also want to create syntax highlighting for VS Code, a linter, etc. It has come a surprisingly long way in such a short while, and I’ve learned so much Python because of it. I’m also learning a good amount about C. I’m hoping once I create this, there will never be any reason to use C over Sea. There are reasons why certain languages aren’t used in certain scenarios. However, I see no reason why certain syntaxes are limited in the same way. Making indentation a part of the language forces developers to write more readable code while removing characters to type. Languages should be made more simple, without compromising on functionality. That is my goal.

Sprint 1 Retrospective

Summary

In this sprint, I mainly did the following things:

  • Learn Scrum – We all had to review how Scrum works and the different roles.
  • Understand GuestInfoBackend – I read through the Architecture to understand how the GuestInfoBackend is supposed to work.
  • Review Docker – I did research and found useful links to help us remember how to use Docker.
  • Docker Example – I created a basic docker example that we can go back to as a reference.
  • Learn Express – I did research into Node, Express, and OpenAPI for the backend.
  • Express Example – I created a basic express example with a few routes and such.

What Worked Well

I think overall, all of us in our team are all on the same page about this class. We understand what we have to do and no one is falling too far behind or going too far ahead. We’re good at helping each other when we need it and we try to put the team first.

Learning how cards work on the boards went really well I think. After that sprint, I think we all have a decent understanding of how to add labels, assign people, etc; moreover, we understand how to utilize the cards to aid workflow.

I also think we overall did a decent job of researching the materials we needed to and we have a decent understanding of the tools involved such as Docker, Vue, OpenAPI, etc.

What Didn’t Work Well

The biggest difficulty comes from the fact that this is a class rather than an actual job. That means we only have 3 hours of class time per week, and then we need to allocate personal time to work. I find this especially challenging when I have other homework to do, and other personal projects I want to work on such as creating a version of the C programming language that has python-like syntax. The biggest limiting factor to all of this is time, with the second one being motivation/interest. I find it incredibly easy to spend an entire day working on something I really enjoy and I find it incredibly hard to work for even half an hour on something I don’t care about. This falls somewhere in between the extremes.

Another thing that adds difficulty is, at times, it feels like we have no idea what we have to get done. The problem is that when that happens, we don’t even know where to start asking questions to figure out where to go.

What Changes Could be Made to Improve as a Team

Overall, we could probably work together more both in class and outside of class. Most of our work has been done mostly alone unless necessary, however there are some times when it would be convenient or useful to work with more people at once.

We all could do better at managing time, as previously mentioned. Using card due dates, we can help keep each other working in a decent time frame.

What Changes Could be Made to Improve as an Individual

I could be more vocal in classes, as well as work more outside of class. This applies to both working more on this class in my own time, as well as working on my other classes in my own time.

Apprenticeship Patterns – Revision

Context

The following post will be in reference to Apprenticeship Patterns: Guidance for the Aspiring Software Craftsman by Dave Hoover and Adewale Oshineye chapter 1 and the introductions to chapters 2 to 6.

This revision is to decrease word count.

Opinion

“Most people won’t have an opportunity to work in a formal apprenticeship where they are being mentored by software craftsmen. In reality, most people have to claw and scratch their apprenticeships out of less-than-ideal situations. They might be facing overbearing and/or incompetent managers, de-motivated coworkers, impossible deadlines, and work environments that treat novice developers like workhorses, storing them in small, rectangular stalls with a PC and a crippled Internet connection. (Apprenticeship Patterns)”

I have many options for what to discuss, so I’ll be discussing the current state of the job environment. Due to numerous factors, students are graduating college and finding that, in order to get an entry level job, they need experience. In order to get experience, they need an entry level job. It’s a catch-22.

Every issue is infinitely complex. There are numerous variables that have caused this situation to exist and it would be impossible to adequately address all of them. So, I will focus on a few such as the abundance of college degrees. Just as every issue is complex, every good thing has bad aspects and every bad thing has good aspects. Most people can agree that a population with college educations is a positive. Education provides opportunities. However, this is bound to also have unforeseen consequences.

Employers want to hire people that will be the best for the position. This is essential for the survival of the business and the economy at large. Not everyone is fit to be mathematician, for example. So, employers must discriminate between candidates in order to determine those who are best fit for the position. One large determining factor is a college degree. Having a degree in a subject implies familiarity and some level of expertise in said subject. However, it doesn’t end at employers. Those applying for positions realize the advantage of a degree and so an entire generation of parents have been pushing their kids to get a degree, with the goal being a decent job.

This drive for degrees creates a huge supply of degrees and with the demand remaining roughly the same, the value of a degree decreases. If everyone has a degree, an employer can’t use it as a discriminating factor. So, they move onto others such as prior experience. The problem arises from the lack of supply of jobs and the huge competition for those jobs. We find that thousands of students are going to college now to get a degree they might not even use and end up in massive debt.

This is by no means a simple issue. There are a handful of helpful practices that can improve the situation. Pushing trade schools as well as colleges can lead kids down a successful path and decrease the supply of degrees, making them worth more. Government can decrease regulation to allow more small businesses to compete for workers. However, there will be no magical single solution to this problem and what it really requires is a societal discussion about college degrees.

Work Cited

“Apprenticeship Patterns.” Accessed February 22, 2021. https://learning.oreilly.com/library/view/apprenticeship-patterns/9780596806842/#toc-start.

Apprenticeship Patterns

Context

The following post will be in reference to Apprenticeship Patterns: Guidance for the Aspiring Software Craftsman by Dave Hoover and Adewale Oshineye chapter 1 and the introductions to chapters 2 to 6.

Opinion

“Most people won’t have an opportunity to work in a formal apprenticeship where they are being mentored by software craftsmen. In reality, most people have to claw and scratch their apprenticeships out of less-than-ideal situations. They might be facing overbearing and/or incompetent managers, de-motivated coworkers, impossible deadlines, and work environments that treat novice developers like workhorses, storing them in small, rectangular stalls with a PC and a crippled Internet connection. (Apprenticeship Patterns)”

This is an incredibly true statement. Due to the current state of the economy, it is very hard for people to gain work experience in any field, let alone in software development. One will frequently see requirements for years of experience in an entry level position, but how can one achieve years of experience if every entry level job requires it? In my opinion, this is the product of many things; moreover, it is an incredibly complex issue, as all issues are. One such thing is the abundance of college degrees. As with many of these possible causes, they are often both positive and negative. Every good action has negative consequences and every negative action has positive consequences.

The positive that comes with the abundance of college degrees is likely self-evident – average people are educated and have more opportunities. However, consider the issue of student loan debt. Many people put themselves into massive debt to get a degree they might not even use. Anyway, it is basic economics at play; you cannot fight supply and demand. The hiring process is simple. Now, the world we live in isn’t perfect but suppose it were an actual meritocracy where everyone has equal opportunities. How can we hire? If we assume that someone who has spent years in college dedicated to studying a topic we work with, we can discriminate based on degree in order to discriminate on skill set. It is important to note that discrimination is not inherently bad. In a meritocracy, discrimination is necessary; in fact, in any value hierarchy where object A has more value than object B, discrimination is necessary. The problem with discrimination is when we discriminate based on irrelevant characteristics. For instance, race has no influence on a person’s ability to be a good doctor. So it is unfair to discriminate based on race when deciding a doctor. However, suppose you need brain surgery and you have the choice of a doctor with twenty years of experience, or a new hire without any experience. In that case, in order for society to function at all, it must be considered fair to discriminate based on experience since that directly affects the chance of success and, in this case, survival. Thus, jobs fairly discriminate based on perceived ability and understanding to find the best for the job. However, the more people that have a degree, the less worth a degree has relative to employment. Having a degree is less of a discriminating factor and so jobs have to create other means of determining skill – hence the absurd amount of experience required.

It is a catch-22; employers use college degrees to discriminate to determine those best fit for their position so average people push their kids to go to college. Then, college degrees become worth less but people still think they are necessary so despite the immense cost, they push their kids to go anyway. Obviously, college is not bad. However, their is a failure to have the necessary discussion with kids. They have to decide whether the career they want is going to be worth the investment. They also need to consider that they might not even find a job in their field after college. It would help the issue if parents and schools viewed college as more of a tool rather than a necessity, and if they introduced kids to trades. Trades also aren’t for everyone, but they are a very founded career path. What society needs overall is a balance and a discussion. In my estimation, the entire education system needs an overhaul; but that is for another blog post.

I would go into the other things I think are causes of such a job environment, but I’ll save that for another day. Instead, I’ll focus on the immense power of environment. First and foremost, people like to view their environment as separate from their minds. However, just think about it. As an example, when you’re lazy, you create clutter in your environment and when your environment is cluttered, it can cause stress or discomfort. Your environment affects your mood and your mood affects your environment, so the distinction between the two isn’t as clear as people might like to think it is. A good working environment is crucial to doing good work. Think back to primary or secondary school and think about the old books and uncomfortable desks with pencil carvings in them. It might go without saying but, if you want to improve your work, put some effort into ensuring your work environment is designed for you to be productive.

Work Cited

“Apprenticeship Patterns.” Accessed February 22, 2021. https://learning.oreilly.com/library/view/apprenticeship-patterns/9780596806842/#toc-start.