Play Station 4 Online Co-Op without PS+?

A PS+ subscription is generally required to play online, however "free to play" type games are exempt from this rule. Currently there are only a handful of games; Warframe, Airmech, PlanetSide2, etc. Please note that PS+ provides more than multiplayer and those features are still disabled for the previously mentioned games.

The Collection you talk about includes “Borderlands 2” and “Borderlands: The Pre-Sequel”, both support “couch co-op” aka “split screen” or “local multi-player”. This mode does not require PS+ obviously, but with the help of “Share Play” one player can join from a remote PS4. (Although some games (i.e. Borderlands 3, not sure about the ones you ask about) explicitly disallow this, for unknown reasons.) The player inviting the other remote player needs PS+.

Even though this play mode has different names, what it always has in common is: the game is executed on one console only and you only need one copy of the game. Consequently, you have only one screen (or the same image on multiple screens).

This is in contrast to “online multi-player” (a not-so-fitting name here as well), where each player runs their own copy of the game¹ on their own console with their individual screen. This mode usually requires PS+ since the PS4 generation of consoles. At least “Borderlands 2” has this game mode as well (according to Wikipedia).

The exact extend of activities which require PS+ in online-multiplayer varies: apart from promotions (e.g. “Red Dead Redemption 2” had once 2 weeks PS+-less online multi-player), games like “Watchdogs (2)” have an asymmetric 2-player mode where one player sneakily enters the single player session of someone else. The “victim role” does not require PS+ to experience/participate. The online-sharing in “Death Stranding” requires no PS+ either.

_{¹ just fyi: with the primary/secondary trick one “license” of the game can be installed and run on two systems simultaneously}