Yes you can do that with a template, but are you looking for just a simple template which you supply the link names for, or something smarter?

If the later, you need to get clear rules about how to determine the links based on the current page's name or something, usually it's tad complicated because it's not always exactly "Chapter 1" "Chapter 2" etc.

Something like this is is also often integrated into the infobox, like for example:

For episodes:

For events: