Does internet speed or computer speed affect the FPS of the server?

Probably not.

The server does not render the game; it merely gives information to the client about the game world around it. While setting the view-distance option could theoretically affect it (clients will only receive data about the world up to that point), in most cases, it won't affect it.

If this is the issue, there is a few things you can do. You can adjust the video settings (such as render distance, particles, etc). This may make the game look not as nice, however, it'll run faster. Also consider installing the Optifine mod.

However, if you are talking about the game speed, it is a whole different ballgame.

How "fast" the server runs is up to two different things:

1. Network connection speed

If the connection between a client (or multiple clients) and the server is slow, due to there being either a long distance or a myraid of other issues from ISPs failing to connection throttling on the server-end to what have you, the game will appear to lag to the clients.

While the game itself is running the world completely fine, there may be issues for the clients connected such as mobs or other game elements not seeming to update as quickly, and sometimes people can lose connection to the server itself (it might show up as User[/0.0.0.0] disconnected: Timed out on the server console).

If this is an issue, the only way to fix this is for the server to get a better network connection, the player to get a better network connection, or to move closer to the physical server's location.

2. The server computer's can't keep up

This issue applies when the game can't run at the correct speed. In normal conditions, the game runs as 20 ticks per second. In a tick, there are many things that happen (random block updates occur, redstone pulses (every other tick), TNT explodes, mobs do AI stuff, time passes, and so on and so forth). When the game can't keep up, it runs under this magical 20 TPS number, and things just happen slower, regardless of the connection to the server.

Things like hundreds of thousands of entities (even simple ones like dropped items) in a loaded chunk can bring the server speed down to a crawl. The server may output a warning to the console that takes the form of this:

[WARNING] Can't keep up! Did the system time change, or is the server overloaded? Running 9001ms behind

If this is an issue, having a faster computer is going to be the best bet here. Except for strict chat, the vanilla (and Spigot and CraftBukkit) server runs on one thread, so having a faster clock speed is going to be more valuable than many cores. Additionally, you can try to modify the view-distance and spawning settings you are using Vanilla, in addition to adjusting certain gamerules (like randomTickSpeed). While certain functionality may be nerfed or lost this way, the server will certianly run faster.

You can also do much more customization with Spigot (out of the box) and some plugins for the Spigot server, although this requires some technical knowledge (as you have to build the server yourself).

The important thing to note that this is all separate from FPS, which is how many game frames the client can render. You can have the game running at 120 FPS, and 10 TPS, and things will happen twice as slow (even though the actual video that the game displays appears smooth) or the game running at 10 FPS and 20 TPS, and the game will feel very very laggy (even though as far as the server (and probably other players) is concerned everything is fine).