1      General Lay Summary

This research is based on the assumption that students are frequently and continuously using Personal Technology (PT), such as mobile phones, tablets and laptops, during lecture time and the fact that they use them does not mean that they do so for purely academic reasons. Does this usage cause any problems either during the lectures or impact on the students’ general performance? How do other students, or the academic staff, view this usage? These are just a small sample of questions which will can help answer the research question of “Whether technology can be disturbing in lectures”.

In order for the research question to be answered, a preliminary research was undertaken, which involved the collection of a small randomised sample of data from students at the Faculty of Technology of the University of Portsmouth, using an online questionnaire created on Google Forms, face to face interviews and observations were performed during lecture time. All the observations, questionnaire and interviews were designed and performed, with respect towards my fellow students, academic staff and their need for anonymity and data accuracy.

The initial findings show a relation between the gender of the student and the usage of technological devices during lecture time, the preliminary results indicate that females use significantly less PT than male students. Students also tend to group together in lecture theatres, having all the students that use technological devices sitting near each other. Students that keep their devices on the desk, tend to use their devices with increasing frequency as the lecture lingers on, in comparison to the students that keep their devices out of sight, for example in their bags. The vast majority of the students periodically checks their mobile phones though none of the students used their mobile phones for academic purposes such as taking notes. A relation between the students predicted grades and their technology usage has been identified.

To validate the initial results as well as to provide more conclusive and robust results, the need for a sufficient population sample and the collected data is identified. Therefore, this research will involve a larger overall sample of students from the entire University of Portsmouth, using stratified sampling. The sample will contain an equal number of students per faculty, year, and gender and the further selection will be randomised to avoid sampling bias.

The research will be carried out during the period of 2017 to 2020, and the data collection will be concluded in twelve intervals, beginning and end of autumn and spring semesters of each year. This will ensure that the research includes comparable data, minimize further bias in sampling and validate the results of each iteration.


2      The Description of the Proposed Research and its context

2.1     Background

The majority of today’s students are digital natives who have been interacting with technology from a young age and arguably, any attempt to capture their attention with traditional teaching methods can be a challenge (Yong & Gates, 2014, p. 102). As personal technologies (PT) become more prevalent, many University students are taking PTs in lectures with them. Students will have their PT with them at all times of the day and so taking PT in lectures is now considered normal. However, the devices do not always remain in students’ bags or pockets and lecturers have noticed that students are interacting with their PT during class. This research is trying at ascertain whether the use of PT within a classroom environment can support or detracts from student learning.

To address this problem different Universities are taking differing strategies to deal with the situation. Some Faculties within universities are banning the use of PTs within classes whilst others are leaving it to individual lecturers to make this decision. (Tindell & Bohlander, 2012, p. 7). However, different rules for each class can cause confusion amongst students and often lead to “irritated students”, as to what they are allowed to do. (Jackson, 2012, p. 130) For example, Strauss (2014), banned the use of PTs during his lectures, due to the high level of student distraction caused by the excessive use of PT by the “same students as before the digital era” and he claims that students are willing to pay attention but the PT usage does not allow them to do so. On the other hand, students that are digital natives have different educational needs, “as they learn and think differently”, when compared to the students of the pre-digital era (Yong & Gates, 2014, p. 102). This would be an indication that PT causes more diverse and higher generation gap between today’s students and their teachers.

The main research topic area of this study, which is to identify relationships between the technology used in the lecture room for students and the disruption this may cause, as well as the effects that this use might have in the learning styles, student disengagement and the impact on the student grades. We investigate the student’s perception of their use of PT within the classroom and whether they are using the technology to support their education Ragan, Jennings, Massey, & Doolittle (2014, p. 78), Prestridge (2012, p. 451).

The main cause of concern is that students’ that are using PT equipment for non-course related activities during lecture time, are possibly minimising their learning experience and opportunities (Alsaggaf, Hamilton, & Harland, 2013, p. 2). A regulated approach to PT researched by Fried (2008, p.912) concludes that unless a faculty requires the use laptops regularly via an integrated into the curriculum way, they should consider ways to limit their use, since the results of his research, are showing that a higher percentage of students that use their laptops during lecture time, have a lower understanding on the course subject and their overall performance is significantly decreased. The same is concluded by Ragan, Jennings, Massey & Doolittle (2014, p. 85)  which is supported by their findings that more than 60% of the student’s laptop usage time is spent on non-lecture related subjects, as well as that student’s attention declines after the first quarter of the lecture. Student engagement with PT is also explored by Junco (2012, p. 197), who tries to identify the reasoning and motivation of the students when they are using PT for non-course related topics, and the negative effect that might have on their overall performance.

The effect of student disengagement from lectures and the negative effects this might have on the overall student’s performance, is identified by Sana, Weston, & Cepeda (2013, p.25) as the difficulty and detrimental effects that any human experiences whilst performing two or more tasks at the same time, known as multitasking (Barak, Lipson & Lerman, 2006, p. 257).

Generally speaking, pen and paper students often have better performance than the students that were multitasking using technological devices. (Wood, Zivcakova et al., 2012, p. 371-372) The same results were presented by Strauss (2014), identifying that multitasking on a laptop causes lower results on both users and fellow students.

Diminishing your own learning opportunities is a personal choice and arguably acceptable, however, being disrespectful during lecture time and disturbing the learning of others either by multitasking or due to movement and light produced by PT screens, is something that cannot be tolerated.  (Sana, Weston, & Cepeda, 2013, p. 25) The studies mentioned above, to my knowledge, are either focusing on only one type of PT, for example, laptops (Fried, 2008) or they are measuring the effect of only one software or application such as Facebook (Roblyer, McDaniel, Webb, Herman, & Witty, 2010) or they are having a really small target audience of 44 university students (Sana, Weston, & Cepeda, 2013, p. 25). Therefore, the need for broadening the scope which includes all forms of PT, measured on a sample of students, which includes at least an entire faculty, for example the faculty of computing of University of Portsmouth, is considered a necessity.


2.2     Research Hypothesis and Objectives

2.2.1     Research Idea and Research Aims

In recent years technology has become a ubiquitous element of higher education and ownership of technology such as smart phones, tablets and laptops has become prevalent amongst university students. At the same time academics are concerned about the lack of engagement with their courses by students. Part of this problem appears to be that students are distracted from engagement with the class by the use of personal technology. This research tries to ascertain the scale of the problem and whether or not students perceive this use of mobile technology as having an effect on their studies. Research was conducted at the University of Portsmouth and involved observation, questionnaires and interviews to questions the true impact of technology and engagement from a socio-technical perspective. It is a qualitative, in-depth exploratory study using examples of experiences and attitudes. The research study will be framed by an in-depth literature review to examine our findings against previously conducted studies. The research is limited to lectures held at the University of Portsmouth and does not include seminar or practical sessions.

This research seeks to identify relationships between students and their use of technology in the lectures. The research addresses the issue from a sociotechnical standpoint in evaluating the effects on student engagement and student learning. The following specific aims are explored:

  1. In what fashion do students use personal PT equipment in lecture theatres?
  2. Do students claim that experience distraction or loss of concentration or loss of attention from PT equipment?
  3. What are the students’ perceptions and practices regarding the use of PT?

2.2.2     Preliminary results

A case study was performed based on the same research aims, objectives and methodology as the proposed research, during January-March 2016, at the University of Portsmouth, Faculty of Technology students.

The results indicate a relation between the students predicted grades and their technology usage; with students that expect to score a high grade, for example a 1st (more than 70%), to use laptops more, than students that expect to score anything below 2:2 (less than 59%) who are using mobile phones more often and usually for non-lecture related tasks.

There was a diversity of student opinions regarding the effect the use of PT has to their concentration. There seems to be a trend on different opinions between female and male students, (Figure 1) which is consistent with the lower PT usage from female students during the observation stage.

A relation between students’ gender and their PT usage has also been identified, with female students using PT dramatically less than their male colleagues. (Figure 2)


Figure 1 Negative concentration effect per Student gender


Figure 2 Average of PT users per 10 minute intervals and gender

The final results of this proposed research will include:

  • Quantitative data collected during observations
  • Quantitative data collected from the Questionnaire
  • Qualitative data collected from the Interviews
  • Qualitative data collected from Questionnaire

2.2.3     Programme and Methodology

Mixed-methods of research are gaining popularity and becoming the standard term for research cases that involve quantitative and qualitative methods. Mixed-methods include the need for different methods to be used in different stages and for various purposes during the research (Glogowska, 2011, p. 253-254). For example, using material from observations to edit qualitative interviews and finally to help construct a quantitative questionnaire to support the interview findings. Every research method has its limitations and different approaches may complement each other. The pragmatic approach (mixed-methods) provide this research the opportunity to use any method, technique and procedure from either quantitative or qualitative research design. The choice of mixed-methods provide the concept of triangulation, which denotes the use of one method to explore and evaluate a concept, which a second or third method will be used to act as verification of the initial results. (Online, 2009)

In this study, the proposed approach is the methodological triangulation (Figure 3), which includes the use of multiple methods to study a research problem. Results from observations in lecture theatres, interviews and questionnaire will be compared to see if the results are similar. If the conclusions from each of the three methods are the same or similar, then the research has established validity. (Online, 2009)


Figure 3 Research Methodology

2.2.4     Target Population and Host

The research will be hosted at the University of Portsmouth. The results of the proposed study will include a sufficient sample of all students in all Faculties at the University of Portsmouth. The proposed sampling method is stratified random sampling (Stroud, 2016).

To collect the stratified sample, all the students will be divided to Faculties, then to separate courses and then to male and females. After the categories have been formed, a set number of each category’s students will be randomly picked, for example 10 students from the Faculty of Technology, which study computer science and are female.

2.2.5     Observations

The observations will be performed by the research team, will last for two and a half years, performed twice each semester at the beginning and end, to cover the diversity of student behaviour depending on their stress levels; they will be conducted in the lecture theatres, including students from the target population. The observations cover the following questions:

  • How prevalent is the use of personal technology (PT) within the lecture environment.
  • What are students interacting with on their PT during lectures?

The observations will be carried out in a quantitative context and will involve:

  1. Counting the total number of students
  2. Count how many students use laptops, tablets, mobiles or other technological devices
  3. Count how many students keep their mobiles on the desk
  4. Count how many students that use a technological device, engage in the lecture either by answering or asking questions or in any other form
  5. Count how many students that are not using a technological device, engage in the lecture either by answering or asking questions or in any other form
  6. Count how many students distract or disturb others, due to their device usage, even if they do not realise, for example in case of snooping.

The high number of data, which we hope to be able to gather, would provide the research with higher validity and usability in a uniform and complete manner.

2.2.6     Interviews and Questionnaire

The interview format will be structured and conducted by the research team in a face to face format. The questions will be asked with the same manner, having the same order and where possible they will be phrased in order that a limited range of responses may be provided.

The Interviews and Questionnaire cover:

  • Students’ perception towards their use of PT within lectures.
  • Students’ perceptions of others, use of PT within lectures.
  • Students’ perceptions of the effects of PT on their ability to engage with lectures.

The interviews will include interviewees from the target population, who will be questioned anonymously and in an ethical manner. The questionnaire will be published to the target population, and it will be designed to include measurable questions, which result in quantitative data. (Learning, 2009)

The vast majority of the questions will be closed, fixed-choice or Likert scale, where participants are given a range of options, for example, disagree, strongly disagree, etc. (‘Likert scaling’, 2006). To avoid participants to be forced to answer a question in a manner that does not represent them, the closed and fixed-choice questions usually provide the user the option “Other” or “I do not know”, in case the user does not wish to answer the question, or has a different personal opinion. There will also be, a few open ended questions that result in qualitative data.

The data gathered from the interviews and the questionnaire, will initially be organised, which includes data transcription, cleaning and labelling; the data will then be coded, structured and defined, due to the need of grouping the data to provide us the best results possible in an efficient and acceptable manner to analyse. (Haregu, 2012, p. 35-37)

2.2.7     Analysis

The majority of the interview questions result in quantitative data, which can be analysed to provide us with statistics and graphical representations, similar to the analysis of the data that were gathered from the observations. However, there will be a sufficient number of qualitative questions, which were asked in the same informal manner, encouraging the interviewee to feel free to answer as if we had a friendly conversation and not formal interview.

The analysis of the qualitative data that both the interviews and questionnaire will provide, follows a deductive approach and it is performed in two ways. The deductive approach is chosen over inductive, due to time constrain and the need of grouping the data to find similarities and differences. (Haregu, 2012, p. 25)

Firstly, the data will be collectively used as to describe the situation of the use of PT within lectures from a social perspective. This category contains data from questions that fall under the ideal category, such as: “In your opinion is it “socially” normal to have your technological equipment with you, active at all times; for example during a family dinner?” (Haregu, 2012, p. 19).

Secondly, patterns will be attempting to draw from the interviewee answers that relate their gender, predicted grade and use of PT devices in lectures with their perception and feelings towards the use of PT from others in lectures and the effects that this use may have on their ability to concentrate and engage with the lectures.


2.3     Project Management

2.3.1     Research team


This project will be managed by me with the help of my experienced supervisor, Dr Penny Ross. She will provide all the feedback mentioned in Section 4.1 as well as guidance and mental support.

The project will include two MRes students as research assistants, to help with the observations, interviews and questionnaire, since they already have some research experience and would benefit more chance to convey the required research, therefore they will be keen to produce accurate results. The milestone that will affect the research assistants, is the “conduct research (Including all intervals)” one, which can be seen at Section 4.1.

The proposed research team will also include a sociology PhD or equal researcher, who will assist in the analysis of the results from a sociological perspective. This will be especially helpful during the stage of completing the middle chapters and conclusion as can be seen at Section 4.1.

2.3.2     Additional Resources


The additional resources of this research include:

  • SPSS software licences for me and the research assistants, to analyse the research results;
  • A microphone to record the interviews, if the interviewee consents;
  • Microsoft Word licences for the entire team, so as we can share the working document; however, if all the team members are from the University of Portsmouth, they already have it;
  • Google drive accounts for all team members, to be used as a tool for collaboration and safe storage of digital material;
  • Notebooks and other pen and paper material;
  • Access to laptops or desktops, email addresses and mobile phones for the entire team, to enhance communication and productivity.

2.3.3     Risks and Limitations

Apart from the expected time and resource constraints that apply to any research project, below there is a table that identify further potential risks, their impact and includes mitigation plan.

Table 1 Risks and mitigation

Risk description and type


Risk impact – description


Risk probability (severity x likelihood)


Mitigation / control


Security – Network loss – Work loss If the University Network fails, research data might be lost Extremely severe – Likely Auto-save functionality in MS Word to be enabled.
Human – Refusal – Not available Members of the target population might not be willing to fill in the questionnaire or attend the interviews or provide consent. Very severe – Extremely Likely The randomised methodology will allow us to pick another student from the same group.
Security – Data/Report Loss – Personal Device faulty If a computer crashes, a lot of research data and results might be lost. Extremely Severe – Likely Frequent updates and backups in the cloud, at least once every 4 hours of work.
Human –  disappointed or feel ethically violated because of the interviews, questionnaire or observations Students might act differently during the research stages. Extremely Severe – Very Likely Explain very well the research motives and make sure that the ethical form is complete and up to date. Also request signed consent from the interviewees.
Health and safety Personal and health related issues with one or more research team members. Extremely Severe – Likely Frequent communication and regular updates on the situation. A team member might need to be replaced if the absence is prolonged.

2.4     Anticipated Outputs and Outcomes

Predicted outputs of this research is a book of PhD standards and an academic paper based on the methodology of this research.

Predicted outcomes of this research include personal development, teamwork, new collaborations and potentially a new theory based on the result of this research.

3      Impact Summary

3.1     Beneficiaries and Impacts of this research

Below is a list of identified beneficiaries of this research either from private or public sector:

  • Lecturers and the teaching staff in general

This is the foremost group of masses that if they decide to induce from the outcomes of this study, will cause the residual of the groups mentioned here to benefit the quickest and the most. Teaching staff can really implement the results of the proposed research at their own lecture rooms, therefore they will benefit from better student averages, better student performance and more up to date teaching style and that, will help students benefit too.

  • Students and their families

This research will enable students to act as informed citizens and may inspire the next generation of researchers, due to student engagement, either as research assistants or as members of the target population (Section 2.2.4). It will likewise provide the students and their families a better value for money on the students’ learning experience, since the University fees are expensive.

  • Universities

The proposed research will increase the effectiveness of lectures and as a result the students may achieve better scores and overall performance; hence the University services will provide better value for money for the students, their families and the wider public.

  • UCAS and Student finance UK

UCAS and Student finance will benefit equally due to the possible growth of the quality of studies, which is possible to contribute to higher wages, more literate and possibly overall more capable teachers, and in general better the life of the students and their ability to repay the student loans.

  • Wider public,

Higher student engagement and more up to date teaching methods, may lead to higher educated students, which may lead to better employees overall.


3.1.1     Timescale and Likelihood

The likelihood of the potential impacts depends on the Universities perspective as to include the research results into their routinely lectures or not. Furthermore, it depends on the appreciation of the results by the Academic community and if they will commit to a possible proposition for lecture changes. The government and the wider public may also have an input related to the likelihood of the potential impacts, due to the fact that the impacts depend on the general public approval and as a result a Governmental approval.

The impacts might start being visible by the first four years after this research is complete and published, due to the fact that the majority of UK students need 4 years to successfully finish a University course, included any gap year, placement or time to travel.

3.2     Academic impact

Academic advances, across and within technology and sociology disciplines, including significant advances in understanding, methods and theory as they are stated in section 2.2.1.

4      Diagrammatic Work Plan

The diagrammatic work plan illustrated in Section 4.1 includes the work of all research team members, combined in one generalised Gantt chart.

The initial milestone identified in Section 4.1 is the point that the draft bibliography will be identified and the research scope is already set. At this point the two research assistants will join the team until almost the end of the research project that includes the milestone namely “conduct Research”.

The two research assistants will initially undertake the task of helping with the literature review as reviewers. Following that, they will be conducting observations and interviews on the target population as described in Section 2.2.3.

The analysis of the results from all three of the data gathering methods (Section 2.2.3) will be performed after each observation interval, for example, at the beginning of one semester, and will be initially performed by the two research assistants. However, the closer we get to the “Conduct Research” Milestone, the stronger analysis will be required, and approximately at Year 2 month 7, the sociology researcher will join the team, to provide the necessary sociological perspective on the data analysis.

Around the same time that the sociology researcher will join the team, the initial chapters and introduction of the book will be written until Year 2, month 10. Those chapters will include the literature review until that point of time, the introduction, methodology, research aims and objectives chapters as well as a chapter that will mention the results of the case study that was mentioned in Section 2.2.2. One year later the middle chapters that will include the main body of the book will be completed. The middle chapters will include the data gathering description, the data analysis, and the produced results and suggestions of the team, regarding the proposed research.

Finally, milestones are identified near the end of the research project, that include the completion of the conclusion chapter at least one month before the project’s final deadline, so as there is time for extensive proofreading and last minute changes on the wording of the chapters. Roughly the same time the secondary literature reading will be finished, to ensure that all the latest academic papers are included in the final output (Section 2.4)


4.1     Gantt chart

The diagrammatic work plan of this research is best illustrated on the Gantt chart below.


Figure 4 Gantt chart

5      Reference List

Alsaggaf, W., Hamilton, M. C., & Harland, J. (2013). CS students’ readiness and perceptions of using mobile technology during lectures. Retrieved February 21, 2016, from

Barak, M., Lipson, A., & Lerman, S. (2006). Wireless Laptops as Means for Promoting Active Learning in Large Lecture Halls. Journal of Research on Technology in Education, 38(3), 245-263.

Barry, S., Murphy, K., & Drew, S. (2015). From deconstructive misalignment to constructive alignment: Exploring student uses of mobile technologies in university classrooms. Computers & Education, 81, 202–210. doi:10.1016/j.compedu.2014.10.014

Chen, G. D., Chang, C. K., & Wang, C. Y. (2008). Ubiquitous learning website: Scaffold learners by mobile devices with information-aware techniques. Computers & Education, 50(1), 77–90. doi:10.1016/j.compedu.2006.03.004

Conole, G., de Laat, M., Dillon, T., & Darby, J. (2008). ‘Disruptive technologies’, ‘pedagogical innovation’: What’s new? Findings from an in-depth study of students’ use and perception of technology. Computers & Education, 50(2), 511–524. doi:10.1016/j.compedu.2007.09.009

Fried, C. B. (2008). In-class laptop use and its effects on student learning. Computers & Education, 50(3), 906–914. doi:10.1016/j.compedu.2006.09.006

Glogowska, M. (2011). Paradigms, pragmatism and possibilities: mixed-methods research in speech and language therapy. International Journal Of Language & Communication Disorders46(3), 251-260. doi:10.3109/13682822.2010.507614

Haregu, T. N. (2012, March 6). Ashok macha. Retrieved March 10, 2016, from

Hong, F.-Y., Chiu, S.-I., & Huang, D.-H. (2012). A model of the relationship between psychological characteristics, mobile phone addiction and use of mobile phones by Taiwanese university female students. Computers in Human Behavior28(6), 2152–2159. doi:10.1016/j.chb.2012.06.020

Jackson, L. D. (2012). Is Mobile Technology in the Classroom a Helpful Tool or a Distraction?: A Report of University Students’ Attitudes, Usage Practices, and Suggestions for Policies. International Journal of Technology, Knowledge & Society, 8(5), 129–140.

Junco, R. (2012). Too much face and not enough books: The relationship between multiple indices of Facebook use and academic performance. Computers in Human Behavior, 28(1), 187–198. doi:10.1016/j.chb.2011.08.026

Learning, I. of L. (2009, November 12). 7. Developing a questionnaire. Retrieved March 10, 2016, from

Ledbetter, A., & Finn, A. (2016). Why Do Students Use Mobile Technology for Social Purposes during Class? Modeling Teacher Credibility, Learner Empowerment, and Online Communication Attitude as Predictors. Communication Education, 65(1), 1–23. doi:doi:10.1080/03634523.2015.1064145

Likert scaling. (2006). Retrieved March 10, 2016, from

Lindquist, D., Denning, T., Kelly, M., Malani, R., Griswold, W. & Simon, B. (2007). Exploring the potential of mobile phones for active learning in the classroom. Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education, Retrieved from:

Online, V. (2009, August 21). Alzheimer Europe – research – understanding dementia research – types of research – the four main approaches. Retrieved March 5, 2016, from

Park, S. Y., Nam, M., & Cha, S. (2012). University students’ behavioural intention to use mobile learning: Evaluating the technology acceptance model. British Journal of Educational Technology, 43(3), 592–605. doi:doi:10.1111/j.1467-8535.2011.01229.x

Prestridge, S. (2012). The beliefs behind the teacher that influences their ICT practices. Computers & Education, 58(1), 449–458. doi:10.1016/j.compedu.2011.08.028

Ragan, E. D., Jennings, S. R., Massey, J. D., & Doolittle, P. E. (2014). Unregulated use of laptops over time in large lecture classes. Computers & Education, 78, 78–86. doi:10.1016/j.compedu.2014.05.002

Roblyer, M. D., McDaniel, M., Webb, M., Herman, J., & Witty, J. V. (2010). Findings on Facebook in higher education: A comparison of college faculty and student uses and perceptions of social networking sites. The Internet and Higher Education, 13(3), 134–140. doi:10.1016/j.iheduc.2010.03.002

Sana, F., Weston, T., & Cepeda, N. J. (2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education, 62, 24–31. doi:10.1016/j.compedu.2012.10.003

Spiegel, A., & Rodríguez, G. (2016). Students at university have mobile technologies. Do they do m-learning?. Procedia – Social and Behavioural Sciences, 217, 846–850. doi:10.1016/j.sbspro.2016.02.006

Strauss, V. (2014, September 25). Why a leading professor of new media just banned technology use in class. Washington Post. Retrieved from

Stroud, D. J. (2016, April 14). Basic sampling strategies: Sample vs. Population data. Retrieved April 14, 2016, from

Tindell, D. R., & Bohlander, R. W. (2012). The Use and Abuse of Cell Phones and Text Messaging in the Classroom: A Survey of College Students. College Teaching60(1), 1-9. doi:10.1080/87567555.2011.604802

Wood, E., Zivcakova, L., Gentile, P., Archer, K., De Pasquale, D., & Nosko, A. (2012). Examining the impact of off-task multi-tasking with technology on real-time classroom learning.  Computers & Education58365-374. doi:10.1016/j.compedu.2011.08.029

Young S. & Gates P. (2014). Born Digital: Are They Really Digital Natives? International Journal of e-Education, e-Business, e-Management and e-Learning, 4(2), 102-105, doi:10.7763/IJEEEE.2014.V4.311

Leave a Reply

Your email address will not be published. Required fields are marked *