While the loading speed in it self is very important for user experience and now Googles ranking. The mechanisms behind it, other than brute server speed, are very important as well. With mobile now being so important average speeds on desktop can be far to slow for mobile users and they will leave in disgust.
**Here are just a few quick tips to help steer you in the right direction for better SEO results:
Nothing slows a website more than errors in the code. These can be broken links to pictures, css or .js files or local server files. Servers work really to load all files, so if you have your server/browser on a wild goose chase looking for stuff that is not there it'll slow everything down. It can mean the difference of several seconds or more just for one file.
The same thing applies to your inside your .css and .js files. Strip them down to the bare mininium. If you need one fontawesome icon, don't settle for the easy option and load all 400+ of them. Minifying .css and .js can be great as well. Just make sure you leave the non minified copies in the folder for the person following behind you.
Note: Where possible more your Javescript to the bottom of your page. It is far better to have it load last than first.
Not resizing images for the page is really poor practice that can kill performance. I have seen 5000px pictures being used for thumbnails before. I personally like to crop the picture about 10% smaller than my max size. This normally keeps it looking nice and saves some bulk. Additional to this I will compress the image with something like minijpeg or my new favourite Kracken. Compress the picture to death. I am not avocating everyone has pixelated/blur pictures, but sometimes you don't need magazine quality everywhere. Save the bulk on the background that doesn't matter and use the savings to have a .svg of the logo instead.
Last tip always use the most efficient file type. For example .jpegs can be compressed many more times smaller than a .png file. So if you don't need the transparent background use a jpeg.
Note: A good server will compress the images server site if you specify the image sizes in your code.
Reducing server requests is a good idea to improve performance. So if your using bootstrap or some common files. See if you can server it from a cdn service such as bootstrapcdn or Googles hosted libraries. This will reduce server requests from your server and provide faster load times....in theory :)
Don't assume these services are optimised or honest. Most especially the free ones are at least in part malware and could also be slowing down your website. Inspect them and use with a measured approach. Often they aren't offering you much benefit.
Google analtics and others take a fair while to load. If you place it at the top of your page it most likely will cause the whole page to load slow or atleast after it loads. If it is at the bottom of the page at least the user experience will appear faster.
Also we often use two analytics, except when performance is hurt. There is no point measuring how fast users dump your page because your four analytics programs are slowing it so much.
With all things performance, don't just read something and assume it will work. Always test and measure. Create a baseline and make improvements one at a time. Happy website page tuning everyone!