To assess the relevance and quality of the source, several factors should be considered. For instance, check whether the articles in a journal have been peer reviewed and whether the document follows a recognised academic format. It is also important to reflect on the purpose of the text, where and how you found it, the author's background, the quality and tone of the language used, etc.
Consider this
-
Scientific information
Scientific information is typically characterised by the fact that it is published in scientific publications by researchers within various subject areas. Different types of scientific publications include, for example, articles, conference papers and theses.
You might also need to use other types of sources in your studies, such as information from authorities and organisations.
Disinformation and misinformation
Disinformation is false information that has deliberately been spread with an aim to influence and mislead the receiver. Misinformation entails information that is incorrect or false but where the sender is not aware of this. Read more about disinformation and misinformation on the University of Borås website.
About misinformation and disinformation at The American Psychological Association (APA) webpage.
A critical approach
To distinguish between different types of information, you should adopt a critical approach. When working with scientific publications and other sources used in your studies, it is important to understand the differences between various types of publications and to know how to evaluate and assess the material you intend to use. Keep in mind that you should be able to explain your reasoning during your information-seeking process and why you chose the sources you did.
In today’s information society, it can be difficult to know which information can be trusted. The Swedish Psychological Defence Agency has a glossary and other matieral that explain concepts such as desinformation, misinformation, propaganda, source criticism, media and information literacy, propaganda, and freedom of expression.
The Psychological Defence Agency also has the website Don't be fooled - Get the tools! where you can read more about how you can increase your resilience against disinformation, misinformation, and propaganda.
-
Scientific journals in which scientific articles are published have clear guidelines concerning the type of research to be published and how submitted manuscripts are reviewed by experts. This is known as peer review or referee review. In the Ulrichsweb database (external link), you can search for journals and check if peer review has been applied. The review procedure is usually available on the journal’s website.
Similar demands are required of other types of research publications, such as doctoral theses and certain conference papers. The conference website will state which contributions have been through the peer reviewing process.
In many databases, it is possible to limit a search for material from publications that have been peer reviewed. Remember, however, that even if a publication has a peer review procedure, it is only the scientific articles that are peer reviewed. A scientific journal contains various types of material, for example, letters and editorials; these are not peer reviewed.
-
In many fields of research, scientific documents commonly follow the IMRaD structure, which stands for introduction, method, results, and discussion. This is often reflected in the document’s structure and headings. In some disciplines, the letter "a" in IMRaD represents analysis. Additionally, many research papers include a theory section. Most scientific studies also include an abstract at the beginning and a list of references at the end.
Abstract
Scientific articles often include an abstract, i.e., a summary of the article’s contents; this is usually placed at the beginning of the article.
Introduction
In the introduction section, the subject is to be clearly presented. In connection with this, there are often definitions of concepts and a research base for the problem at hand that is grounded in previous research. Theories that may form a base for limitations and perspectives may also be included here. The introduction also includes the aim that specifies the type of conclusions that the study may expect, followed by one or several specific research questions.
Method
In the method section, the approach used in the data compilation is motivated and accounted for. This is done irrespective of whether the study is based on empirical material (such as interviews, surveys, observations, measurements) or is a systematic review of previously published research. If the work is a systematic study of the literature, then there should be an account of how the information search was conducted.
Results
The results of the study are given based on the approach that was presented in the method section. The results form the basis for the answer to the research question or questions which were presented in the work. Depending on the type of investigation, the results section may include tables, charts, pictures or interview quotations.
Discussion
The discussion section leads to the conclusions based on the results of the study.
References and Appendices
Throughout academic work, there are clear references to all the sources used. All sources that are referred to in the text are to be given in the list of references at the end of the work. Finally, any appendices are usually found at the end.
-
A primary source can, for example, be an original article in a scientific journal, a conference paper, a research report or a thesis. In an original article, research results are presented publicly for the first time. Whenever possible, you should refer to primary sources when citing research results.
Course literature is typically not a primary source, but a secondary source in which research findings are presented or summarised. Popular science publications also fall under secondary sources. Secondary sources are based on primary sources and provide interpretations, analyses, or summaries of them. An example of a scientific secondary source is a review article.
-
The contents of publications
General questions to consider regarding the content of documents and publications:
- What is the aim or purpose of the document?
- Are the contents based on facts or are they to create opinion?
- Is the author’s perspective objective and impartial?
- Are other sources clearly and consistently given?
- Is the language factual and correct?
- Does the document have a clear structure?
- Who is the intended target audience?
Questions to consider about the contents of scientific publications:
- Is it clear how the study was conducted?
- Are the results and conclusions reasonable?
- Is there anything controversial?
- Was the study commissioned or funded by someone with a potential interest in the outcome?
Time
Certain subject areas, for example, within the natural sciences and information technology, have higher demands on a source being relatively recent. Other areas, such as the humanities, can use older sources. Note that the time between a researcher conducts a study and the time it is published can be long, in certain cases, several years.
- When was the document published, is it recent enough?
- Is the method used still current?
- Has a lot happened within the area since the study was conducted?
-
Publisher
A publisher may be a publishing company, an organisation or an authority.
- Which publishing company has published the journal/book?
- Is the publishing company specialised in publishing scientific material?
The Nordic lists can be used to check the scientific value of a publication channel, for example, a journal, conference, or publishing company. Remember that you must also evaluate the specific article or conference proceedings themselves, even if the journal or conference is included in one or several of the lists.
- Norwegian list (external link) (journals, series, publishing companies)
- Finnish list (external link) (journals, series, publishing companies, conferences)
Source of information
- Where was the document accessed?
- Which databases and search engines have you used?
- Are there scientific contents in the databases?
- How are the search hit lists ranked, which hits come first?
Indexing of journals
Is the journal included, is it indexed, in a database? In the Ulrichsweb database (external link), you can search for journals and see in which databases they are indexed.
Doubtful journals and irresponsible publishing
There are journals and conferences that are less serious than others. The University of Borås has developed a guide for assessing doubtful journals and conferences.
Paper mills are a dishonest form of publishing scientific articles where the aim is to make money from researchers who pay to have an article written and then published in a scientific journal. Articles produced by paper mills are often characterised by being AI-generated or plagiarised from scientific material.
If you find an article marked as 'retracted,' it means that it was published in a journal but has since been withdrawn. The reason for the retraction could be that it was discovered the article was produced in an dishonest way, for example through a paper mill. A retraction can also happen if it is found that something was wrong in how a study was presented, a mistake that made the author decide to retract the article.
-
The author(s) of a publication
To gain a deeper understanding of the material you are reading, it is helpful to investigate the author more closely by considering the following questions:
- Who conducted the study or wrote the book?
- What is the author's background; has the person any connection to a department or organisation, so-called affiliation?
- What are the values and objectives of the department or organisation the author is associated with?
- What has this person published previously within the area?
- What is the person's expertise related to the subject matter of the article/book?
In Scopus and Web of Science databases it is possible to search for publications and get information concerning the authors’ affiliations and previous publications. It is also possible to get information about what a researcher has published in Google Scholar and on the ORCiD website.
H-index
An H-index is a measure of how much a researcher has been cited in relation to the number of publications the researcher has written. A researcher’s H-index can vary depending on the source used to calculate it (for example, Web of Science, Scopus, or Google Scholar), which is due to differences in the coverage of these databases.
It is not advisable to compare different people’s H-indexes with each other as the H-index depends, for example, on how long a researcher has been active and which field of research the researcher belongs to. Therefore, it is also not possible to define what constitutes a high or good H-index.
-
Impact factor
The Web of Science database includes the impact factor, which is based on how much a journal’s articles are cited during a specific period of time. The results are presented in ranking lists based on the subject area. The lists are divided into quartiles in which a journal can appear in the first, second, third or fourth quartile depending on its impact. The highest ranked appear in the first quartile.
Note that it is not possible to compare impact factor between journals in different subject areas as the subject areas have different traditions for citation. The impact factor does not include quality of a specific article, each article should be evaluated independently.
In Scopus, there is a similar factor, SCImago Journal Rank Indicator (SJR).
Citations
In the Scopus and Web of Science databases, it is possible to see an article’s citations. A citation shows that others researchers have observed a certain publication. A publication can be of interest for various reasons. One reason can be to show the quality of the publication, and another can be to discuss weaknesses in the quality. It can also be done to show knowledge of a study, or for other reasons.
-
Research funding
Questions to consider regarding the funding of a study:
- Who or what has funded the study being reported?
- Is the funding public, for example from The Swedish Research Council (Vetenskapsrådet), the Swedish Knowledge Foundation (KK-stiftelsen), or Region Västra Götaland?
- Or is the funding private, such as from a company or organisation?
- Is it possible that the funder(s) may have influenced which results the researchers presented?
- Are there any ideological motives and/or economic interests that could have affected the study?
Ethical issues in assessing publications
All scientific publication has high demands on openness and honesty. Specific ethical demands are included when the research concerns people and animals. Consequently, there are ethical guidelines and committees for certain research areas. Below there are links to some of the most important. These can be useful when reflecting on ethical considerations while assessing articles and other research publications.