Mordax

Software Developer

View My GitHub Profile

Pulling JSON from a URL data value

Rust is definitely a much more difficult language than I initially thought. I didn’t expect to run into a huge difference between pulling a key that already had a displayable value like the language key versus going to that key’s data value that’s a url and grabbing the JSON from there.

If that was confusing, basically, the Github API for grabbing starred repositories has a lot of keys in which the value is simple - you can just display it:

Example of Starred repo JSON

description: "Resumes generated using the GitHub informations"  
fork: false  
url: "https://api.github.com/repos/resume/resume.github.com"  
forks_url: "https://api.github.com/repos/resume/resume.github.com/forks"  
keys_url: "https://api.github.com/repos/resume/resume.github.com/keys{/key_id}"  
....etc

The description and fork keys have easy to grab values, for example.

But it also has a lot of keys in which the value is a URL such as url, forks_url, keys_url and other keys. If you go to that URL, you get an unstructured key-value JSON object:

JSON displayed by going to a URL value expressed in the Starred repo

JavaScript: 19277  
HTML: 15229  
CSS: 7329  
Ruby: 179  

supernova doesn’t know what languages the repo will have because that’s all dependant on the starred repo in question. And we can’t predict what languages there will be, so we can’t use keys like in the previous JSON structure.



Rust really tries to keep you safe and from doing dangerous things on a systems level, which from my C/C++ background is quite difficult to adjust to. I like doing potentially ridiculous things with my code and then fixing it before anyone has seen what I’m doing. rustc, Rust’s compiler, felt all of a sudden like a suddenly talkative (and critical) acquaintance hovering over my shoulder.

I’m very used to just playing around aimlessly and enjoying the Eureka moment from realizing how to go about a problem. Rust is not one of those languages. Granted, I had to get this work done for the end of my semester and I was in a rush, but this is a language that requires a lot of care and respect. You need to take that Rust Programming Language book lying on your bed and read it front to back.

On top of that, this is the first time I’m working with pulling information from an API. There’s quite a bit to learn in terms of how deserialization and serialization works, how to properly format JSON objects, how to use them in a meaningful way and do all of this in a language I’ve never worked with, with concepts I’ve never experienced before. So unsurprisingly I’ve had quite a few moments of frustration.

I do think this is worth writing about, I feel that I’ve learned quite a bit from this currently not working pull request.

Issue:

Using the Github API to grab programming language information on the starred repositories.

Solution (But mostly side issues):

This is my first ever Work In Progress [WIP] pull request. It’s disheartening to put up work that isn’t working. However, it is actually really nice to get detailed feedback from others, especially if you’re unable to find solutions online or get feedback via IRC. Getting properly directed and seeing what the maintainers are thinking in terms of design especially on such a young project is great.

Challenges:

Most likely this will get even more complicated because logical progression is to allow the user to request this data when they want to via argument flags.

Also, the information passed in the value url JSON formats the amount of a programming language in bytes. Meaning there needs to be work to convert it into something more coherent.

As well, it could be great to have future enhancements where you can type a programming language you want to use and have the starred repos utilizing those pop up. But this in particular is speculation on my part.

Benefit:

Here’s the Pull Request.