APOD Archive page seems broken

[tauparsec]

It's been broken for me for a while now, probably a couple months. I just wanted to bring it to your attention that I think it may need to be revamped or paginated or something.

Basically what happens is when I go to apod's picture of the day, which works great, I usually go to the archive to go through some of the days I missed and go back through it's catalog looking for neat data on distant objects as that's where my fascination lies the most. Lately I haven't been able to do that. It locks up on my browser and says the page has become unresponsive. This is ongoing. I am currently using Chrome.

If I use another browser, for instance Firefox, I get the infamously ambiguous "Internal Server Error" which means you gotta go through your logs to see what's up. I hope there's a fix coming through the pipeline. I love apod's archive page. It's a great repository for astronomical tidbits and mind boggling photos.

[geckzilla]

This has been previously noted and discussed here:
viewtopic.php?f=28&t=12939&start=275

Also, the editors are aware, and I believe reworking it is either planned or in progress. Uh, I wouldn't guess at when that might be done, though.

[tauparsec]

This has been previously noted and discussed here:
viewtopic.php?f=28&t=12939&start=275

Also, the editors are aware, and I believe reworking it is either planned or in progress. Uh, I wouldn't guess at when that might be done, though.

Ok, good to hear they're aware. Is it possible to open source the code so people can contribute to it?

[geckzilla]

This has been previously noted and discussed here:
viewtopic.php?f=28&t=12939&start=275

Also, the editors are aware, and I believe reworking it is either planned or in progress. Uh, I wouldn't guess at when that might be done, though.

Ok, good to hear they're aware. Is it possible to open source the code so people can contribute to it?

They've received a lot of offers in the past and either declined them or didn't like the results. I'd expect the pattern here to be the same, but I suppose it's not technically impossible...

[tauparsec]

This has been previously noted and discussed here:
viewtopic.php?f=28&t=12939&start=275

Also, the editors are aware, and I believe reworking it is either planned or in progress. Uh, I wouldn't guess at when that might be done, though.

Ok, good to hear they're aware. Is it possible to open source the code so people can contribute to it?

They've received a lot of offers in the past and either declined them or didn't like the results. I'd expect the pattern here to be the same, but I suppose it's not technically impossible...

Interesting. What's nice about having the codebase in a repo like github is that there's configurations for peer reviews, pull requests and the likes. All changes would be transparent and unwanted changes can simply be rejected. I think an open source collaborative environment would speed up development time while utilizing people's free time and free labor. I think it would be a good idea. Regardless I'm ok with whatever the powers that be decide to do on their development timeline and milestones.

[Chris Peterson]

Ok, good to hear they're aware. Is it possible to open source the code so people can contribute to it?

They've received a lot of offers in the past and either declined them or didn't like the results. I'd expect the pattern here to be the same, but I suppose it's not technically impossible...

Interesting. What's nice about having the codebase in a repo like github is that there's configurations for peer reviews, pull requests and the likes. All changes would be transparent and unwanted changes can simply be rejected. I think an open source collaborative environment would speed up development time while utilizing people's free time and free labor. :p: I think it would be a good idea. Regardless I'm ok with whatever the powers that be decide to do on their development timeline and milestones.

I don't think there is a "codebase" as such. Pretty much the entire APOD system is made out of static webpages. The pages aren't generated by any sort of engine, they're hand written. And you can examine any of them directly. So they're already "open source" in the sense of the source being publicly accessible.

[tauparsec]

They've received a lot of offers in the past and either declined them or didn't like the results. I'd expect the pattern here to be the same, but I suppose it's not technically impossible...

Interesting. What's nice about having the codebase in a repo like github is that there's configurations for peer reviews, pull requests and the likes. All changes would be transparent and unwanted changes can simply be rejected. I think an open source collaborative environment would speed up development time while utilizing people's free time and free labor. I think it would be a good idea. Regardless I'm ok with whatever the powers that be decide to do on their development timeline and milestones.

I don't think there is a "codebase" as such. Pretty much the entire APOD system is made out of static webpages. The pages aren't generated by any sort of engine, they're hand written. And you can examine any of them directly. So they're already "open source" in the sense of the source being publicly accessible.

I thought it may perhaps be driven by some kind of java backend or maybe even python since there is an xml feed.

Unfortunately with that kind of layout, it's difficult to make any kind of structural change without normalizing the data in some way, much less try and give it a facelift. That would be a time consuming effort without automating it somehow, and if you're automating it, you're already going through steps of normalizing the data anyway so why not just shove it in a database.

Anyway, I'll wait for them to get around to it, I'm missing that page quite a bit. I will say though, after your input, I can't help but be really really really tempted to scrape the entire site.

[geckzilla]

I will say though, after your input, I can't help but be really really really tempted to scrape the entire site.

I strongly advise against it. I made a fully php/mysql version years ago, but they didn't like it because they didn't want to learn php, didn't want to have to deal with updating, and were afraid I'd abandon it or that php would become obsolete. They like the plain html because it's all they really understand, and they think that it will last the longest and never become obsolete.

There is *some* automation involved, but I forgot what.

[bystander]

Anyway, I'll wait for them to get around to it, I'm missing that page quite a bit.

You might try using the calendar instead of the archive. Maybe it will give you some of the functionality you desire. Clicking on any of the months will give you thumbnails of the images for that month.

[tauparsec]

I will say though, after your input, I can't help but be really really really tempted to scrape the entire site.

I strongly advise against it. I made a fully php/mysql version years ago, but they didn't like it because they didn't want to learn php, didn't want to have to deal with updating, and were afraid I'd abandon it or that php would become obsolete. They like the plain html because it's all they really understand, and they think that it will last the longest and never become obsolete.

There is *some* automation involved, but I forgot what.

Hmm... well I'm not here to redo their site. I'm fine with whatever the powers that be decide to do, and I am tempted to just redo the whole thing too but really I just want to scrape it so that I can have all the content logged myself with all the pics so I can peruse at my leisure without having to worry about the site's infrastructural issues.

BUT, if I were tasked with redoing it with your input in mind, I'd say Python is a better bet. First it's used scholastically and it's backed by NASA so that should be enough. Python is the fastest growing general purpose programming language. Some kind of slimmed down homebrew templating system or already working framework wouldn't be too much of a mind stretch. Plus, since Python is used in the field of astronomy rather extensively from what I've gathered, I don't think it would be too difficult to get a lending hand (for free maybe even) to keep things pruned in case the original programmer disappeared into the ether. Python ain't goin' nowhere. You can get a lot done with a lot less code. The architecture would need to be documented though, and it should be simple enough that you could throw an intermediate python coder to the fire without too much worry.

I think they've just settled with their filesystem as their "database" essentially. There's obvious value adds in normalizing the html files into a database schema that we both would agree on, but that would take coding html out of the equation for them when making a new APOD entry for the day. I wouldn't choose MySQL though, I'd say PostgreSQL or SQL Server depending on their machine. They're fine with their files though so I'm cool with it. I just want them to fix that issue.

[tauparsec]

Anyway, I'll wait for them to get around to it, I'm missing that page quite a bit.

You might try using the calendar instead of the archive. Maybe it will give you some of the functionality you desire. Clicking on any of the months will give you thumbnails of the images for that month.

That looks good! I will be using that for a while.