I built a single-page web app (RepoRecon) that integrates collections of Github repositories across a range of topics that might be of interest to people here:
Awesome lists (34,000 repositories of resources for various topics)
The repositories are presented in a paginated table that you can sort and filter. Each row of the table has a link so you can go to the homepage of the associated repository.
Let me know if you have further topic suggestions, or you can raise an issue .
You can also fork RepoRecon and host your own set of topics. Just edit the topics.json file and build the database locally. Then push it to your Github repo and enable Github Pages and anyone can see it.
That’s pretty neat! I poked around in the KiCAD section for a few minutes and ended up with a pile of bookmarked projects I want to go through in detail.
Do you think it would be possible to let people generate their own topics on the fly without forking? Or are the table entries generated by hand? Being able to type in a few random keywords and get a summarized list of matching repos would be super useful.
That’s the thing: a lot of interesting projects appear to have taken a ton of effort but have like three stars because they can’t be found. My motivation with RepoRecon was to make serendipitous discovery of these projects more likely.
Each topic requires a scan of Github repos using a keyword (like “kicad”) and then collecting all the results into a database (actually, just a JSON file). The search API for Github is limited in terms of what you can search for and rate limited as to how many search results you can get per minute. The initial scan for KiCad took several hours to get 25,000 potential projects. Then I scan every night for any new projects which only takes several minutes. But I don’t know any way to do it on-demand. Also, I do the scanning locally and push the updated data to my Github repo. I don’t know how (of If) Github would allow running the scanner on their site.
You can do that within a particular topic. In the KiCad topic, for example, you could search for projects mentioning “RP2040” and “keyboard” within their project descriptions with the filter string desc:RP2040 && desc:keyboard.
I added Arduino to the list of topics. This gathered over 140,000 repositories that mention Arduino in some way. I can see the web app straining at this number of repos so some changes may be needed.