Review and evaluate
-
Peer review
Peer Review
Scientific journals have clear guidelines for which research is published and how submitted manuscripts are reviewed by experts. This is known as peer review. The database Ulrichsweb allows users to search for journals that have been peer reviewed. Peer reviewing is usually also detailed on the journal’s website.
Similar requirements are also placed on other types of research publications, such as doctoral theses and certain conference papers. Conference websites may state which contributions have been reviewed.
In many databases, searches can be restricted to materials from peer-reviewed publications. However, bear in mind that even if the publication has been peer reviewed, certain parts may not have been reviewed such as letters or editorials.
The Nordic lists
The Nordic lists can be used to verify the scientificity of a publication channel, such as a journal, a conference or a publisher. The lists are based on assessments carried out by groups of experts. If a publication channel is listed at level 1 or above, it should have a certain scientific quality. Please note, however, that you must also review and evaluate the actual article or conference paper, for example, even if the journal or the conference is included on one or more of the lists.
- Norwegian list (external link) (journals, series, publishers)
- Finnish list (external link) (journals, series, publishers, conferences)
-
Structure of scientific documents – IMRaD
In many research fields, it is common for scientific documents to have an IMRaD structure, meaning that it consists of an introduction, method, results and discussion (IMRaD). This is reflected in the structure and the headings.
Introduction
The subject should be presented clearly in the introduction. In connection with this, concept definitions and a background to the problems based on previous research are often provided. Any theories that form the basis for limitations and perspectives may also be presented. The introduction states the aim, which explains which types of conclusions the study is expected to result in, often followed by one or more specific questions.
Method
In the method section, the approaches used for data collection are reported and justified. This is done whether the study is based on some form of empirical material (e.g. interviews, survey responses, observations or measurements) or reports a systematic review of previously published research. If the work is a systematic literature study, there should be a report of how the information search was carried out.
Results
The results of the study are reported based on the approach presented in the method section. The results form the basis for the answer to the question(s) posed in the work. Depending on the type of study, the results section may include tables, diagrams, images or interview quotations, for example.
Discussion
The discussion leads to conclusions based on the study’s results.
References and appendices
Throughout an academic work, there should be clear references to all sources used. All sources referred to in the text should be detailed in a reference list at the end of the work. Any appendices are usually added right at the end.
-
Primary and secondary sources
A primary source could be, for example, an original article in a scientific journal, a conference paper, a research report or a thesis. In an original article, research findings are presented publicly for the first time. You should ideally make reference to primary sources when referring to research findings. Course books are often not primary sources, but secondary sources in which research is reproduced. Popular science works are also secondary sources.
-
Source criticism
When reading texts, it is important that you reflect on the document’s content and background. Below are some examples of questions you can ask yourself to ensure that you are being critical of the source.
Source of information
Where did the document come from? Which databases or search engines were used? Do the databases have scientific content? How is the hit list ranked? Which hits appear first?
Publication
Which publisher produced the journal/book? Is the journal indexed in a database?
Authors
Who carried out the study or wrote the book? What is the author’s background, and are they affiliated with any institution or organisation? What are the values and aims of the institution or organisation? Has the author previously written within this field?
Contents
What is the aim of the document? Is its content factual or opinion-forming? Is the author’s viewpoint objective and impartial? Are other sources reported clearly and consistently? Is the language factual and accurate? Is the work clearly structured?
Does the text explain how the study was carried out? Are the results and conclusions reasonable? Does anything appear controversial? Was the study commissioned by someone with an interest in the results?
Time
When was the document published? Is it sufficiently current? Within certain subject areas, such as the natural sciences, there are stricter requirements for the source to be relatively recent. In other areas, such as the humanities, older sources can be used.
Research funding financiers
Who has financed the study which is accounted for? For example, is it public funding from The Swedish Research Council, the Swedish Knowledge Foundation or Region Västra Götaland? Or is it, for example, private funding from a company or organisation? Is it possible that the person or persons who have financed the study may influence the results which the researchers choose to present? Finally, are there any ideological motives and/or economic interests which may have an impact?
Information concerning the research funding financiers is often available in the respective publication or in the Scopus and Web of Science databases.
-
Facts from other sources
Facts that you may need to reproduce are not always the result of scientific studies. However, facts that are not research findings should still be evaluated with regard to credibility and usability. Ask source criticism questions about the document in order to assess its relevance.
An example might be a White paper (White Paper References - American Psychological Association (APA)).
-
Further information: Is the article scientific?
-
Reflecting on ethical issues when assessing research publications
All scientific publication places strict demands on openness and honesty.
Specific ethical requirements apply when research relates to people and animals. There are therefore ethical guidelines and committees for certain research fields. Links to some of the most important ones can be found below. These may be useful in connection with reflecting on ethical considerations when assessing articles and other research publications.
-
Scientific articles
When you do a literature search you will find different types of documents. You will find scientific articles and non-scientific documents but also maybe scientific documents in various stages of the publishing process. Sometimes different journals can use different names for the same type of material. If you are uncertain, go to the journal’s website for more information. You can also contact the library or your teacher to get more information.
Original articles (original articles, research articles, empirical articles, research papers) comprise empirical studies that give an account of the results of new research for the first time.
Review articles do not report the results of a study of the authors. A review articles describes, analyses, summarises and evaluates published research within a limited field of research. There are various types of review articles with a varying degree of systematic searching, for example, literature review, systematic review, scoping review, meta-synthesis or meta-analysis. A systematic review is a comprehensive survey that is based on a carefully formulated protocol.
There are theoretical articles that discuss and develop the theories of a field of research. These are, however, not as common as original articles and review articles.
Articles in various stages of the publishing process
Different versions of one and the same article can be found; this is because they can become available online during different stages of the publishing process. There are various labels for these articles depending on how far they have proceeded in the process.
Preprints are articles which have not yet been peer reviewed (evaluated by subject experts). Do not use this type of article without checking with your teacher first. Another name that is used is author original manuscript.
Postprints are articles which have been peer reviewed but have not yet been given their graphic form. Sometimes they are called author manuscript, accepted article or accelerated article preview.
Articles that have been peer reviewed, have their graphic form and are published on the Internet but for which there is no decision regarding which journal issue they will be published in, can be called published ahead of print, earlycite article, article in progress, article in press, online ahead of print, early view and online first.
-
Popular science publications vs. scholarly publications
-
Grey Literature
Grey literature comprises various types of literature that are not published by commercial publishers. It is important to note that grey literature can have a varying degree of scientificity from doctoral theses to newsletters. Consequently, scientific literature and grey literature are not mutually exclusive but it is always important to be critical of sources. A considerable amount of grey literature is not scientific, such as reports from authorities, organisations and companies but you can often use them in your introduction and background sections; it is, of course, important that you differentiate between what is scientific literature and what is not.
For more information and examples, see the Library website of the Karolinska Institute regarding “grey literature”.