I implemented a basic Matrix class for additions of matrices, multiplication of them, and multiplying every entry of matrix by constant, operator== and operator!=, overloads for I/O operations using istream and ostream.

Before you start reading it, please look at some rationale about horrible aspects of it:

The reason for naming member functions rows() and columns() was that rowCount and columnCount were already reserved for private member variables, thus I couldn't name them that way. I was thinking between just returning the dimensions from vector itself, but chose the way the code is.

My knowledge about const member functions is not sufficient to be confident, that is why I made the code as messy as it is, but I hope it is safe, at least.

I think that range loops are more explicit and names for the variables are clear enough, but I would like to receive an opinion of you.

According to wikipedia page, #pragma once is supported wide enough to still use it.

\$\begingroup\$About iostreams, it is actually intended! <iosfwd> only declares std::ostream so that it will increase compilation time and user will include their stream if needed, otherwise it won't bring whole iostream. Otherwise, thanks for the review!\$\endgroup\$
– IncomputableJan 3 '17 at 8:07

Well, coderodde already found the bugs, and he's right that you should consider mapping the matrix onto a single piece for compactness.
Still, the underlying storage should be provided by a std::unique_ptr<T[]> instead of a std::vector<T> to avoid redundancy.

The ctor Matrix(size_t rowCount_, size_t columnCount_) should mirror or be implemented in terms of the ctor Matrix(size_t rowCount_, size_t columnCount_, const T& value). The innermost loop is redundant.

The move-ctor should construct a 0/0-sized matrix and then swap everything. There's no appreciable benefit to allowing a zombie-matrix violating matrix's invariants.
Same for move-assignment, make it a full swap.

By the way, where is the non-member swap-function? A custom function can be more efficient and helps in implementing the rest.

The subclass std::domain_error seems to be a better description of the error when the dimensions are wrong for some operation than std::logic_error.

I would reserve rows and columns for a range-object for iterating over them, and use row_count and column_count for obtaining the counts.

The problem with your mutable operator[] is that a user can change the size of the returned row-vector, which is invalid. That's a problem of the representation.

You are missing operator*(Matrix<T> rhs, const T& lhs).

Your stream-extractor can only extract matrices of known dimensions. That's somewhat inevitable given the format your stream-inserter produces.

You should try to consistently insert single characters as characters instead of strings for efficiency, even if they are written as an escape-sequence.

You've already got some good answers, so I'm only going to address one of your design decisions:

The reason for naming member functions rows() and columns() was that rowCount and columnCount were already reserved for private member variables, thus I couldn't name them that way.

This decision resulted in a rows method that consists of a single line, returning the rowCount field.

size_t rows()
{
return rowCount;
}

Without any context, this looks confusing. There are two things to consider here.

Use a naming convention to avoid conflicts

There are various naming conventions that can be used to give variables names. They each have their own followers, benefits and drawbacks. If you use different conventions such as 'snake_case' for member variables and 'camelCase' for method names then it can help to avoid conflicts between them. Alternately, some places will prefix member variables with a known value e.g. 'm_rowCount', or be explicit about get methods 'get_rowCount'. Find a convention that works for you and adopt it.

If you can't work around a conflict, then:

Prioritise the public interface

Your classes public interface is the one that is used by external clients. In many ways it is more important than the private implementation. If you decide that you want to rename a private field, you only have to fix any references to the field in that class. If you decide to rename a public method on the class, you may have to refactor many dependent pieces of code to take account of the rename. If you have to make a decision about which name to get right, make it the public one.