New APOD RSS Feed Required

[OgetayK]

This is a thread to create a new RSS feed for APOD. Currently, APOD's RSS Feed has some flaws. It is our hope that the APODs volunteer-run Asterisk community can create a properly functioning RSS feed. This is an important need, especially for the news platforms such as Flipboard. We try to reach out to as many as possible. Although APOD reaches millions every day, since different types of platforms have different audiences, we must be able to reach all, especially for educational purposes, and avoid possible polarization.

Here are some insights:

• The main page of APOD is updated every day and has this URL: https://apod.nasa.gov/apod/astropix.html

• Also, a clone page is created with this URL for example: https://apod.nasa.gov/apod/ap211114.html

• RSS Feed should only see the links that have the above format (apod/ap211114.html) not the main page with a format (https://apod.nasa.gov/apod/astropix.html)

• The required RSS has to follow these rules: https://ko-kr.about.flipboard.com/rss-guidelines/

Best,
Ogetay

[RJN]

Yes, well stated. In is indeed our hope at APOD that an RSS-savvy volunteer will come forward.
- RJN

[aularon]

Good day,

I volunteer to work on this. I should have a working feed for you to test fairly shortly. I will need some non-technical help and input from your side, about which I will get back to you soon.

I maintain the [Arabic APOD mirror](https://apod.me/), and have some good understanding of APOD content.

Best,
Hasan

[OgetayK]

Good day,

I volunteer to work on this. I should have a working feed for you to test fairly shortly. I will need some non-technical help and input from your side, about which I will get back to you soon.

I maintain the [Arabic APOD mirror](https://apod.me/), and have some good understanding of APOD content.

Best,
Hasan

Thank you for your interest Hasan, we are looking forward to hearing back from you. Meantime, everyone, please always feel free to work with other volunteers as a group or you can work on it individually too. Remember this is not a work dedicated to only one person, one can always work on it until we hit our target.

Ogetay

[Bhavin Joshi]

Hi There!
I get APOD on my Ubuntu desktop. I am from India.
I am interested in volunteering.

Regards.

Bhavin Joshi
https://bhj.im

[aularon]

Good day,

I volunteer to work on this. I should have a working feed for you to test fairly shortly. I will need some non-technical help and input from your side, about which I will get back to you soon.

I maintain the [Arabic APOD mirror](https://apod.me/), and have some good understanding of APOD content.

Best,
Hasan

Thank you for your interest Hasan, we are looking forward to hearing back from you. Meantime, everyone, please always feel free to work with other volunteers as a group or you can work on it individually too. Remember this is not a work dedicated to only one person, one can always work on it until we hit our target.

Ogetay

Thank you, Ogetay. I totally agree with the points you raised, having multiple people contributing to this makes it more rich and sustainable.

I created a script that generates the RSS feed, here is the first generated feed: https://apod.me/en.rss
Next steps:

1. I will clean up the script code and upload it to github under MIT license, so that anyone can see, modify, use and contribute to the code.
2. You, and essentially any one, can report issues / suggest improvements. I personally haven't used an RSS reader in ages.
3. Deployment: While I can host the daily-generated feed files on my server (apod.me), I believe it is better on the long run to have a mechanism to update the feed on apod.nasa.gov itself. This needs further discussion.

Best,
Hasan

[aularon]

Source code: https://github.com/etabits/apodrss (MIT Licensed)

The script was updated to also output other feed formats (Atom 1.0 & JSON Feed 1.0).
The script is smart enough to link youtube's image in case we have a youtube video embed, it also goes the extra mile to fetch embedded vimeo video image and link to it in the feed.

Please anyone feel free to check the code, adapt it, send pull request, raise issues, etc. Also, please let me know what do you think... I want your feedback, positive or negative! I still don't know if it is working for you — Afterwards, process (generation mechanism) should be formalized, as mentioned in my previous reply.

Best,
Hasan

[OgetayK]

Source code: https://github.com/etabits/apodrss (MIT Licensed)

The script was updated to also output other feed formats (Atom 1.0 & JSON Feed 1.0).
The script is smart enough to link youtube's image in case we have a youtube video embed, it also goes the extra mile to fetch embedded vimeo video image and link to it in the feed.

Please anyone feel free to check the code, adapt it, send pull request, raise issues, etc. Also, please let me know what do you think... I want your feedback, positive or negative! I still don't know if it is working for you — Afterwards, process (generation mechanism) should be formalized, as mentioned in my previous reply.

Best,
Hasan

Hi Hasan,

I was making a few tests this past few days. First of all, I really liked the feature where it uses a thumbnail for a video! On Flipboard, posts are also in the historical order which is definitely an improvement for us! Although somehow Flipboard gets the information of the authors and states "Jerry Bonnell & Robert Nemiroff" under every post, they require us to use dc:creator tag. Could you add this future if possible, please? Otherwise, we can not get verified.

Also, I'll ask you to check the other requirements on Flipboard Guidelines for RSS if I miss any: https://ko-kr.about.flipboard.com/rss-guidelines/

I'm seeing that your RSS Feed is not updating itself since Nov 26. Is it possible to update it every day for a while, so we can test whether it works without any problems for now?

Thank you!
Ogetay

[aularon]

Hi Hasan,

I was making a few tests this past few days. First of all, I really liked the feature where it uses a thumbnail for a video! On Flipboard, posts are also in the historical order which is definitely an improvement for us! Although somehow Flipboard gets the information of the authors and states "Jerry Bonnell & Robert Nemiroff" under every post, they require us to use dc:creator tag. Could you add this future if possible, please? Otherwise, we can not get verified.

Also, I'll ask you to check the other requirements on Flipboard Guidelines for RSS if I miss any: https://ko-kr.about.flipboard.com/rss-guidelines/

I'm seeing that your RSS Feed is not updating itself since Nov 26. Is it possible to update it every day for a while, so we can test whether it works without any problems for now?

Thank you!
Ogetay

HI Ogetay,

Before now, I was conducting my tests (and file generation) on my laptop, manually, then uploading the resulting files to the server — Now I set up an automated mechanism so they update each day within ~20m of publishing. (Need to verify that tomorrow morning!)

I'm happy that the overall result is good! I will investigate the remaining dc:creator thing and integrate it. BTW How may I check it via Filpboard myself? Because in the linked requirements document they say that it is rather recommended but not required: (I believe the actual verification is slightly different from the requirements listed)

The following elements are not required, but are highly recommended:

<pubDate>: The date of the article’s publication, in RFC822 format. If not present, will use the time the element was first seen by the Flipboard server.
<guid>: a string that uniquely identifies the item within the feed.
<description>: A short, plain-text summary or abstract of the article.
<dc:creator>: Name of the person who wrote the article. Use multiple <dc:creator> elements for multiple authors.

Thank you,
Hasan

[OgetayK]

Before now, I was conducting my tests (and file generation) on my laptop, manually, then uploading the resulting files to the server — Now I set up an automated mechanism so they update each day within ~20m of publishing. (Need to verify that tomorrow morning!)

Hi Hasan, thank you! Now, this seems to be updating itself and it looks good!

A quick question: Why there are no thumbnails on the RSS? Flipboard still seems to be able to get the thumbnails, but I can't see them on the visualized RSS. Is this a problem, or it is OK? The older version of APOD RSS shows thumbnails too (https://apod.nasa.gov/apod.rss)

BTW How may I check it via Filpboard myself?

You can create a Flipboard account and create a magazine with your RSS feed URL. This will show you the result. Here is the one that I did: https://flipboard.com/@apod_nasa/apod_n-6hle4fq9y

As you may notice, we currently have two problems:

1) The latest post is yesterday's post.
2) There is still no thumbnail for the video.

Because in the linked requirements document they say that it is rather recommended but not required: (I believe the actual verification is slightly different from the requirements listed)

Yes, they are recommendations, but it takes a lot of time to get verified by Flipboard. So want to follow all the recommendations to prevent any possible inconvenience.

Best,
Ogetay

[aularon]

Hello again, Ogetay!

I updated the generator to do the following:

• add a list of dc:creator(s). It extracts them from the copyright line (photographer/Hubble/ESA/etc.) and then appneds both RJN and Jerry to the list
• put the image in the feed! (I had this extracted like I said before, but apparently forgot to add it in the final generated XML)
• add a short description for each item, besides the full content that's already there
• did the required rss validation fixing few issues in the process

Regarding the feed being one day behind, here's what I'm doing: New APODs are usually added at midnight GMT - 5, so I am regenerating the feed around 00:20 GMT - 5 (giving a buffer of ~20m to make sure it is published). Please check at the next cycle (Dec 5th within an hour after publishing) and you should get the new day's item on the top of the feed.

Please check and let me know what do you think.

Best,
Hasan

[OgetayK]

Hello again, Ogetay!

I updated the generator to do the following:

• add a list of dc:creator(s). It extracts them from the copyright line (photographer/Hubble/ESA/etc.) and then appneds both RJN and Jerry to the list
• put the image in the feed! (I had this extracted like I said before, but apparently forgot to add it in the final generated XML)
• add a short description for each item, besides the full content that's already there
• did the required rss validation fixing few issues in the process

Regarding the feed being one day behind, here's what I'm doing: New APODs are usually added at midnight GMT - 5, so I am regenerating the feed around 00:20 GMT - 5 (giving a buffer of ~20m to make sure it is published). Please check at the next cycle (Dec 5th within an hour after publishing) and you should get the new day's item on the top of the feed.

Please check and let me know what do you think.

Best,
Hasan

Hello Hasan,

These updates look great! This project is really coming along. I was watching the Flipboard page that I've created for the last a few days. For some reason at the beginning --after this message of yours-- the last APOD wasn't there. But now, it seems to be working properly. However, the publishing time is 02:00 AM --which is not important in our case.

As far as I can see, everything is functioning properly. I would like you to check this Flipboard page (https://flipboard.com/@apod_nasa/apod_h-b1kcbdp3y), you might be able to see problems that I don't see. The only thing that I've noticed is that for videos, there are still no thumbnails. Could you please look into that?

Best,
Ogetay

[aularon]

Hello Hasan,

These updates look great! This project is really coming along. I was watching the Flipboard page that I've created for the last a few days. For some reason at the beginning --after this message of yours-- the last APOD wasn't there. But now, it seems to be working properly. However, the publishing time is 02:00 AM --which is not important in our case.

As far as I can see, everything is functioning properly. I would like you to check this Flipboard page (https://flipboard.com/@apod_nasa/apod_h-b1kcbdp3y), you might be able to see problems that I don't see. The only thing that I've noticed is that for videos, there are still no thumbnails. Could you please look into that?

Best,
Ogetay

Thank you Ogetay! I'm happy with the progress as well.

I was adding the images in something called "enclosure" for feed items, but apparently it is not the mechanism utilized by Flipboard. I updated it now to work in the way they use — but it needs testing from your side, as I am unable to access the link you provided (maybe it is private for your account only?). The good thing is (from what I understand) is that we should now not only get video previous, but the full video embed! — Also needs checking from your side.

Regarding the updates, there seem to be a mechanism (PubSubHubbub) to notify feed consumers (i.e. Flipboard) when content is updated. I'll see how can that, or an alternative, be implemented, in the next step!

Please check and let me know how things go. Also, if I can check the page in the link you provided, that would be great.

Best,
Hasan

[OgetayK]

Hello Hasan,

These updates look great! This project is really coming along. I was watching the Flipboard page that I've created for the last a few days. For some reason at the beginning --after this message of yours-- the last APOD wasn't there. But now, it seems to be working properly. However, the publishing time is 02:00 AM --which is not important in our case.

As far as I can see, everything is functioning properly. I would like you to check this Flipboard page (https://flipboard.com/@apod_nasa/apod_h-b1kcbdp3y), you might be able to see problems that I don't see. The only thing that I've noticed is that for videos, there are still no thumbnails. Could you please look into that?

Best,
Ogetay

Thank you Ogetay! I'm happy with the progress as well.

I was adding the images in something called "enclosure" for feed items, but apparently it is not the mechanism utilized by Flipboard. I updated it now to work in the way they use — but it needs testing from your side, as I am unable to access the link you provided (maybe it is private for your account only?). The good thing is (from what I understand) is that we should now not only get video previous, but the full video embed! — Also needs checking from your side.

Regarding the updates, there seem to be a mechanism (PubSubHubbub) to notify feed consumers (i.e. Flipboard) when content is updated. I'll see how can that, or an alternative, be implemented, in the next step!

Please check and let me know how things go. Also, if I can check the page in the link you provided, that would be great.

Best,
Hasan

Hi Hasan,

Thank you for the updates! I'm sorry about the inconvenience, the link should be accessible for you now: https://flipboard.com/@apod_nasa/apod_- ... ator_share

What I do here is, every time you update, I delete the old magazine and create a new one with your RSS feed. The reason is, Flipboard is not updating the older posts even you update your RSS feed. So in this way, I can see if there are any differences. But I'm not sure if they somehow store the old posts, even if you delete them. Sometimes these are stored on their side. So if you're sure that this should work --for the video-- we can wait for the next video post and see.

It seems that the videos still don't have a thumbnail --see the Nov 30 post. And I don't know why --maybe because of the thing I explained above. But all the other problems seem to be solved which is great! This project almost come along.

You can also create a Flipboard account with your email, and create a magazine with this RSS feed --no other requirements. So you can simultaneously test your RSS feed too if you'd like to.

Best,
Ogetay

[mdione]

Hi people. a week ago I contacted the editors about this, they sent me here:

I'm no RSS expert (although I once wrote my own client), but I think
for ages the RSS feed was kinda broken:

* The first entry always points to https://apod.nasa.gov/apod/astropix.html ,
even when a dated page (ap{YY}{MM}{DD}.html) exists, which means the entry
changes every day, and (some?) RSS clients get confused with this.

* Entries are not dated, so less data for the clients to resolve the above
issue.

And since Jan 02nd I have seen that:

* Entries have an empty title.

That's all, I hope you can make be fixed. Cheers,


-- Marcos.

[mdione]

More info here viewtopic.php?t=42849

[OgetayK]

The RSS Feed was updated by MTU IT in Feb 2023.

Judy Schmidt has pushed a new update upon Hasan Arous's bug fixes on Aug 20th, 2024.

[OgetayK]

Hasan Arous (APOD's RSS developer) reached out for an issue (a previous fix had been lost). I pulled the changes and the issue has been fixed.