My Experience with Prettybird and PLDI 2023 

		Charles Averill
		
		
		As of writing this, it is Monday, February 6, 2023 at 10:53AM, and I am on the topmost floor of UTD's McDermott Library. This morning I received my rejection notice and reviewer comments for my first self-led paper, Prettybird, a domain-specific language for programmatic font generation. I thought a recounting of my story would be useful for other first-time submitters to read, and interesting to look back on as my career through academia (or not) progresses.
	
		Prettybird was born more out of desperation to publish something in the PL field than a genuine interest in the topic, although I will say that this attitude changed as I learned about the history of digital typography, and experimented with new language features. I'm applying to CS PhD programs this upcoming fall, and having only published one white paper in an unrelated field at this moment in time, I felt it pertinent to "publish or perish". Admittedly, I had recently exited a toxic relationship as well, and needed something to obsess over to take my mind off things.

		And obsess I did! Prettybird's first commit was written on October 7th, 2022, exactly 34 days before PLDI's research paper track submission deadline, Novemeber 10th. I had been brainstorming ideas for DSLs for roughly two weeks at that point. Before I started Prettybird, my previous leading idea was to extend the C Programming Language to add instructions that would generate input for Qiskit - or another Quantum Computing simulation backend - in order to perform quantum computations inside of C without having to exit the language. I was very close to starting development on the idea, before I got a gut feeling that I should check once more that nobody had accomplished this. To my dismay, Oak Ridge National Laboratory published this exact idea roughly a year prior in the form of qcor.

		The first Prettybird commit was written at roughly 4AM on October 7th. I had been cramming with my friend Willie; we had put off our research project at the lab we both worked at at the time, and needed something to present the same day at 11AM. Eventually we realized we would not have sufficient material to present, and cut our losses. While we were sitting around, I had a "eureka!" moment, and realized (incorrectly) that there were no programming languages that could be used to generate fonts (my Google skills must have needed work, because I did not find Donald Knuth's MetaFont during the ~30 minutes that I spent doing discovery). This was my primary motivation for creating Prettybird: solely because it was novel. In hindsight, this was bad science. While sort of valid, the best solutions arise out of problems that people care about. I would later find out that there was more motivation for this tool to exist, thankfully.

		I got right to work writing a parser given a rough language example that I had come up with, and about two hours later I had a very decent parser running thanks to the Lark Parsing Library. This was the first time I had used a parsing library, and I was astonished at how fast I was able to develop with it compared to my previous experience writing parsers from scratch. Lark had its own issues but was mostly good. After pushing that first commit I felt incredibly accomplished, and excited for the future.

		For the next 34 days, I essentially did nothing but work on Prettybird. I knew how close I was cutting it with the deadline, having worked on research projects in the past that took months of time to complete. I neglected my research job (honestly, I don't regret this in hindsight. I was planning on quitting soon anyways as we had not published anything after the year I had spent working there and I was growing impatient. I would be laid off along with all other undergraduate and Master's students at the end of November), my schoolwork, and my relationships with friends and family. I was convinced that my effort would be the boost that I needed to set me apart in applications for graduate programs.

		Later that day, after Willie and I's poorly-received and underdeveloped presentation for our research project, I went to lunch with my club's sponsor and the students that worked in his lab. It was at this lunch that I met Takemaru, my co-author on Prettybird. He was also new to this lab, and had just started a Master's at UTD. He was also very interested in the PL field, had written compilers, and was interested in Quantum Computing. I immediately asked him if he'd like to join the Prettybird project, and he did. This was a crucial moment; although our paper was still rejected from PLDI, it absolutely would have been much worse without Take's input.

		It took roughly 4 weeks to get the compiler to a state I was pleased with, although there were (and still are) some major drawbacks that would not be feasible to complete in the time I had remaining. I started with compiling to the BDF format, a text-based format that could represent bitmap fonts. This worked very well and encouraged me further. Afterwards, I compiled to SVG fonts, which also worked very well. However, my methodology for generating TTF/OTF fonts was just to import the SVG-generated fonts into FontForge, and export them as the desired format. This did not work very well, as there were many bugs involving overlapping SVG curves. However, it worked well enough that I chalked it up to "future work".

		While I was developing the compiler, I was telling my friends and professors about the work. For the first two weeks I got a lot of pushback on the concept as a whole, that it was unfounded, that it was not usable, that although it was novel it was not complex enough to publish. These were frustrating comments to receive because I was very certain that the idea had sufficient merit to publish, and certainly enough merit to exist at all. Maybe I was stubborn, or maybe I really did see some angle of the problem that these people did not. I cannot give an unbiased answer to this question.

		During the last week of the compiler's development, and the remaining 6 days between that day and the submission deadline, I wrote the paper. I made an explicit effort to study other DSL papers published at PLDI in previous years to ensure that the structure of my paper was sound at the very least, and to familiarize myself with the writing style of the community. My primary source for this was FaCT, although I studied a few other papers as well. Honestly, this was much more fun than the actual compiler development. I liked researching the history of digital typography for my Background section. I was very pleased to find that my creation had even more reason to exist than I imagined - any good font development software is crazy expensive! FontForge is free but it's so old, and despite using it for weeks I could not get comfortable with its UI and behavior. I liked drawing up diagrams and writing fancy words and outlining my language grammar and making sure my paper looked visually pleasing. The new PLDI template is 1-column, which made me sad because I liked the 2-column papers from previous years. 2-column papers just seem more "academic", but I couldn't explain why.
		
		As soon as I started writing the paper, I started contacting professors I knew to request preliminary reviews of my paper. While 2 weeks is absolutely a tight shcedule, especially provided that it was in the heart of midterm season, I was very frustrated by the amount of professors that told me they would be able to review and comment on the paper in that time, and then ended up not reviewing or even opening the paper. I was a beggar so I could not be a chooser, but I did feel somewhat betrayed by people I had trusted. Had I received more "Unfortunately I'm very busy these next few weeks and will not have time to adequately review your paper" notices this would be a non-issue.

		In spite of this, I did receive one very useful review from the professor of my Programming Language Paradigms course. He suggested the user study that I ended up performing in the paper, and gave me lots of points on the words I used to describe Prettybird, and general advice about navigating academia as a whole. I greatly enjoyed talking with him after class those few weeks, and I feel better prepared for what lies ahead of me because of those conversations.

		I also received reviews from my parents. These were incredible reviews for me, as the perspective of non-programmers helped me tune my paper to read easier and more clearly. They commented on my figures to clarify their meanings, and they fixed a few embarassing grammar errors that I should know better about. My 12-year-old sister also helped with the paper by testing out the language as I developed it. She had done a little bit of programming before, but functional programming, especially recursion, was new to her and I tuned the grammar using her feedback. Around this time I also posted on Reddit announcing the project in font design communities. I got some conflicting feedback; some people said they wouldn't be able to use it because they don't know how to program, some people said it wouldn't compare to graphical font editors. One review I liked said that even though they personally wouldn't use it, the creation of more tools to aid font design is always a benefit for the community. I wholeheartedly agree. 

		The Wednesday of the week before the submission deadline, I visited a language security professor to talk about the idea. While he did not have the time to read the paper, he listened to my summary of the project and provided very useful commentary. Specifically, he introduced me to MetaFont, a font design language written by Donald Knuth alongside LaTeX. I had told the professor that I was sure my idea was novel, and he walked to the other side of the room and handed me "The MetaFontbook" by Knuth. This was crushing in the moment for me. I had done all of this work, sacrificed so much time, to come up with a solution to a problem that had already been solved! However, I looked inside of the book, and I found the syntax to be revolting. Later I found out that MetaFont had been abandoned due to its high learning curve and inability to generate scalable fonts, two areas Prettybird had already beat it at. One thing the professor said that frustrated me was that PLDI was likely not the place to submit my paper, because I was solving an "engineering problem" rather than a "science problem". This frustrated me because I saw the truth in it, but felt that my improvement on user accessibility was sufficient enough to be "science".

		The following Sunday I put together a small user survey that juxtaposed roughly equivalent samples of Prettybird and MetaFont, along with their outputs. The results overwhelmingly favored Prettybird, although I would later find out that my survey was not well-designed (I'd never done it before). Despite its issues, I believe the survey absolutely improved the state of the paper.

		Having rewritten almost the entire paper after learning of MetaFont, and integrating my survey results, I submitted my paper the day of the deadline. I sighed a sigh of relief, that I had at least accomplished something. The waiting was awful. There was no early rejection, which is why it took 88 days for me to receive results that, in my opinion, should've been released in a few weeks, assuming that reviewers had already been found. In the meantime, I started preparing to teach my Practical Compiler Design course, which took my mind off of the worry of not getting published. While I waited, I considered what kinds of reviews I could get. My biggest fears were that the language security professor was correct, and that I would be rejected on the premise that my paper wasn't "science", or maybe that the reviewers would tell me I wrote a bad paper, making me a bad scientist and a worse person. I knew that second one was unlikely, but fear is a weird emotion.

		And finally, this morning I received my rejection. I was expecting a response on Wednesday according to the conference schedule, so it really was a surprise. I woke up to the email and read the reviewer comments before I even got out of bed. The reviewers considered themselves either knowledgeable or familiar with the domain, and they all gave me a 1 (Reject) for the paper's merit. I would consider each review to be fairly scathing, but nothing they said was untrue. I agree with basically every critique and suggestion they have for improving the paper. Each reviewer said that they thought the domain was interesting and lacking in attention. My least favorite review was easily Reviewer A's:

			> Unfortunately, the submission has some fairly significant shortcomings.  The work is not fully developed, the evaluation is not compelling, and the exposition lacks essential detail.  Even if these were fixed, I'm not sure there'd be enough: while I don't doubt that the tool is useful, it's not clear what novel insights or results the work brings.

	Lay off! I thought this phrasing was fairly harsh, especially considering the other two reviews reading much more pleasantly, despite saying roughly the same things. Reviewer A, I hope you were just having a bad day.

		As of now, it's time to write my responses to the reviewers with Takemaru. I hope that I will get one more round of feedback on my feedback, otherwise I don't understand the point in writing responses for a rejected paper.

		I plan to update this page as I think more about the scenario, again when I submit to PLDI's Student Research Competition, and again when I attend PLDI in the Summer.

		Thank you for reading,
			Charles Averill