Evaluation of the teachers’ digital competences in primary and secondary education in Portugal with DigCompEdu CheckIn in pandemic times

The current pandemic crisis the world is living in has brought new and emerging challenges to teachers, making it essential to acquire digital skills, especially in virtual learning environments and online technologies. In this sense, from the DigCompEdu CheckIn self-assessment questionnaire, validated for the Portuguese population by Dias-Trindade, Moreira, and Nunes (2019), the research presented in this paper aims to identify the most fragile and robust areas of digital skills of primary and secondary education (ISCED) perceived by teachers in Portugal. The quantitative methodological approach emphasizes teachers' perception of their digital skills in three dimensions: teachers’ professional competences, teachers’ pedagogical competences and students' competences and involved 434 teachers from mainland Portugal and the Autonomous Regions. The results allow us to conclude that teachers have an overall moderate level of digital proficiency – level B1 Integrators – and the dimensions pedagogical competences and students’ competences are those where teachers have more weaknesses than in other levels. From a panorama observed before the onset of the COVID-19 pandemic, it is possible to understand their needs regarding work that involves digital technologies at different moments in the teaching activity. The results thus, show the need for teachers to increase their level of digital competence through specific training and the importance of developing public policies that prepare teachers for a more digital school.


Introduction
Throughout this century, and primarily due to the changes taking place in society, which is more and more immersed in a networked digital culture and reality, it is increasingly urgent to rethink the teaching and learning processes. Educational institutions have to face the challenge of setting up hybrid learning scenarios that promote the acquisition of knowledge and spaces appropriate to cross-cutting competences.
Taking this reality into account, the World Economic Forum in 2015 highlighted the crucial role of digital technologies in developing cross-cutting competences and how this is decisive in today's world (World Economic Forum [WEF], 2015). In 2013, the Online Manifest was published following a European Commission project, advocating the end of the distinction between offline and online, using the 'onLife' neologism to refer to a new hyperconnected reality in which such distinction is senseless (Floridi, 2015).
Digital technologies have become wholly part of how one can 'make' education (Selwyn, 2016), but it is important to understand how they can be effectively used to promote an educational change and enhance quality pedagogical practices. More than instrumentally using technologies, "[...] the discussion must focus on its pedagogical impact and on what is perceived as 'good' teaching and as factors promoting quality in learning" (Dias-Trindade & Moreira, 2017, p. 99-100, grifo dos autores).
Thus, today, it is important to rethink education, reflect on new practices where the analog, the physical, the digital, and the virtual are interwoven without distinction, using pedagogical action principles inspired mostly by collaborative, socio-constructivist and humanist principles.
These issues have taken on new characteristics since the COVID-19 pandemic forced governments worldwide to order the physical closing of schools and the shift to full digital education for all education levels.
According to Huang et al. (2020, p. 13), this online education should effectively contribute to "Disrupted Classes, Undisrupted Learning, [...]" following a set of principles, including: "(a) reliable communication infrastructure, (b) suitable digital learning resources, (c) friendly learning tools, (d) effective learning methods". However, these principles are not limited to fulfilling a function when education was provided exclusively in virtual environments, but should be available in any educational ecosystem that used digital environments.
As such, from the DigCompEdu CheckIn self-assessment questionnaire validated for the Portuguese population by Dias-Trindade et al. (2019), the research presented in this paper aims to identify the most fragile and robust areas of each teacher, based on their perceptions, which allows, a posteriori, each one to invest in adequate training to increase their proficiency level to the desired digital fluency. In addition to giving the individual perception of each teacher's digital competences, it also shows us how they feel regarding work that involves digital technologies in the teaching activity, both in preparation and implementation and in working with students to promote their digital competences. This paper presents the results of a study carried out from this questionnaire before the onset of the COVID-19 pandemic and pieces together a picture of the digital competences of Portuguese teachers and the areas requiring greater investment, that is, "[...] bridging the gap and fostering the use of digital differences in education, acknowledging the differences to make specific use thereof and harnessing the potential of these [digital] resources" (Dias-Trindade & Ferreira, 2020, p. 182).

The importance of digital competences and the new reality in education
At the turn of the 20th century to the 21st century, it soon became clear that the world was shifting from an industrial era to a social era, wherein homogeneous education has been increasingly called into question. Hence the need to rethink the educational model, since the speed at which changes are taking place, the evolution of machinery, and the progress they have enabled have been responsible for a great deal of such change, leading to a networking society marked by profound economic changes and encouraging the emergence of new learning paradigms, models and scenarios.
At the same time, digital technologies are evolving at such a rate that, as Floridi (2015) argues, education has benefitted from many opportunities, but at the same time, it runs the risk of losing them. Therefore, both teachers and students must keep pace with the changes and, concerning using resources and digital strategies, to understand the need to integrate them properly in a quality teaching practice.
References to the use of digital technologies in education are often found but, as Montez, Montez, and Aires (2013, p. 282) state, [...] although their use is highly valued, it contrasts with their low use in the classroom; greatly used at home, in preparing classes and teaching materials, and low use of technologies in the classroom, or, when used, students do not use them as much.
The problem is often related to the constant evolution of digital technologies that are not accompanied by adequate training and leading. Floridi (2015) stated that teachers to 'build the raft while swimming', recalling Neurath's words (1959, p. 201): "[...] we are like sailors who must rebuild their ship on the open sea, never able to dismantle it in dry-dock and to reconstruct it there out of the best materials".
Today, due to the pandemic the world is experiencing and the consequent adaptations it has had to make, it has come to the fore that there are alternatives and that societies adapt themselves to correspond to the common good (Santos, 2020). The teacher is responsible for this fluidity approach, for being able to adapt to new times, and even of a certain 'naive confidence', fruit of Lewis Carroll's Red Queen hypothesis ("Now, here, you see, it takes all the running you can do to keep in the same place!" 1 ). And that is why Oliveira (2017, p. 5) refers that "[...] as evolution occurs, organisms must become more and more sophisticated not so much as to gain competitive advantage, but rather to stay alive while other organisms in the system constantly evolve and become more competitive".
So, what is important is that conditions are set up to help pedagogical practices be enhanced through digital technologies. Also, investments must be made in teachers' training to focus on developing skills to develop an education mediated by the digital.
However, in addition to this, and to build an ecological scenario in which everything is structured according to their specific characteristics, teachers must naturally take greater control over a resource and content-related issues, since it is up to them to define what they consider most appropriate for teaching (content) and for getting the message across (resources). On the other hand, taking into account their students' specificities, teachers need to know how to adapt to their characteristics and competences.
In this sense, it is important to know how to design learning environments where all the stages are carefully thought out, justifying what needs to be done, why and how. The use of user-friendly platforms and accessible resources is essential to get a message across and, above all, for the message to be understood and well accepted by its recipients.
The use of a teaching and learning model supported by virtual environments should be geared to the development of virtual communities of learning, that is, "[...] a group of people joined together in cyberspace to accomplish a task and obtain a result, being clearly a specific type of collaborative learning" (Dias-Trindade et al., 2019, p. 150), based on continuous reflexive dialogue among its participants.
These ideas are based on establishing a close relationship between teachers and their students in a constructivist learning model and construct personal knowledge. This idea is close to the Garrison, Anderson, and Archer (2000) model of the Community of Inquiry, which favors a diversity of perspectives and promotes research, criticism and creativity, with the student being responsible for controlling their learning by negotiating meanings with the group (Dias-Trindade et al., 2019).
The preparation of a virtual model of teaching and learning should be based on developing competences and principles such as constructivism, autonomy and interaction, through the development of an educational ecosystem adaptable to the needs and competences of all those involved. Knowing how to use digital technologies in the different stages of the educational process is necessary, both as tools to enhance the learning process and contribute to the quality of learning.
The present moment shows the need for raising the status of the teaching profession and, in particular, for rethinking the practices developed and, above all, how the use of digital technologies can enhance these.
Indeed, the reality of education in 2020 makes it more pressing to assess teachers' digital competences and make them more digitally proficient at making a faster transition from emergency remote teaching situations to more robust and sustained networked digital teaching models.
For this to happen, all those engaged in the educational process must be actively involved and seek to take advantage of the resources at their disposal to enhance the teaching and learning processes, but, above all, teachers must be capable of assessing their digital competences and seek the necessary training to overcome their weaknesses.

The DigCompEdu CheckIn framework
As Dias-Trindade and Ferreira (2020, p. 173-174) state, "[...] by leveraging what digital technologies have to offer to enhance the educational process, one can create innovative digital environments and sustainable learning scenarios, where digital can truly be a window to the world of knowledge". To this end, teachers and students need to know how to integrate digital resources into their pedagogical practices and how they can improve the way they teach and learn.
This fact has also been, since 2005, a key concern of the EU Science Hub. This department has prepared various benchmarks, namely the DigCompEdu, presenting a common European framework for the digital competences of educators, launched in 2017, which states that teachers "[...] need a set of digital competences specific to their profession to be able to seize the potential of digital technologies for enhancing and innovating education" (Redecker, 2017, p. 8).
Simultaneously, the working team has developed an online questionnaire based on the collaboration from several European countries, which teachers can answer to identify their perception of their level of digital competence. More than just listing a set of competences provides users with a report where the teacher is placed in a digital competence level by area and receives suggestions for improving their practices.
This document is an excellent starting point for teachers to identify their weaknesses and, from there, to receive suggestions for overcoming them. Its major difference from other benchmarks: the fact that it not only gives teachers an idea of their digital competence, but also articulates that digital competence with the development of learning, the assessment and development of students.
As we can see, there is an interconnection with the notion of digital proficiency, especially where it specifies that the evolution process within digital proficiency, from literacy to proficiency, requires the ability to perceive not only which resources to use, but also how to use them to achieve best the established goals (Briggs & Makice, 2011). Being aware of these competences is therefore crucial, as stated by Dias-Trindade and Ferreira (2020, p. 183): "[...] being able to identify their strengths and weaknesses helps teachers to determine what steps need to be taken to achieve full digital fluency".
Teachers can thus understand how to integrate technology into their practice, what kind of training they need to take to improve the areas in which they are weaker at the moment, and also helps them understand how to progress to digital proficiency, while always realizing that their knowledge is never complete. Moreover, given this type of knowledge that has evolved rapidly, proficient users know that they are not yet 100% prepared and that their knowledge needs always to be developed.

Research ethics
Although the research project underlying this paper has not been submitted to an ethics committee, it has followed the ethical guidelines in the Ethics Charter published by the Portuguese Society of Education Sciences (SPCE, 2014). Throughout the research, the authors remained vigilant of and have self-reflected on ethical issues, as advocated by (Mainardes & Carvalho, 2019).
Participation was voluntary, and participants could withdraw at any time if they so wished. Furthermore, data confidentiality and privacy were ensured and did not represent any constraint for the participants (Bassey & Owan, 2019;Beardsley, Santos, Hernández-Leo, & Michos, 2019). The research relevance and the results thereof would represent an added value for the education research field, especially due to the constraints brought about by the COVID-19 pandemic and the need to implement measures to qualify teachers for using digital tools in their practice.

The participants
This study involved 434 Portuguese primary and secondary school teachers (International Standard Classification of Education -ISCED) from Mainland Portugal and the Autonomous Regions.
As shown in Table 1, most of the participating teachers are women, 81.8%. Teachers' distribution by department is quite balanced, the most represented being the Language Department, with 24.4% teachers, and the least represented being the Expressions Department, with 12.7%. The mean age of teachers is 50.5, ranging from 32 and 65 years of age.

The instrument
From the self-assessment scale of teachers' digital competences prepared by the EU Science Hub, the work was validated for the Portuguese population (Dias-Trindade et al., 2019) to assess teachers' digital competences in Portugal.
The scale, built and validated, has 21 items that allow the identification of six areas of competence, more specifically: Area 1 -Professional engagement; Area 2 -Digital Technologies and Resources; Area 3 -Teaching and Learning; Area 4 -Assessment; Area 5 -Empowering learners; Area 6 -Facilitating learners' digital competence (Dias-Trindade et al., 2019) (Figure 1).
The questionnaire provides teachers with an overall result that places them into six proficiency levels ( Table 2). Partial results for each framework area and a report on the overall result and suggestions for improvement in each of the 21 digital competences referred to in the questionnaire are shown (Table 3).
In this way, teachers can understand which area of competence they have the most weaknesses and choose the training that best meets their needs (Dias-Trindade et al., 2019).
In order to verify whether there are significant statistical differences when analyzing results by age ( Figure  2) and by department (Figure 3), the Wilcoxon Signed Rank nonparametric test was performed. The level of statistical significance adopted is p = 0,05.

Results and discussion
The results obtained show that the overall average was 43.4 points, corresponding to the B1 -Integrator level. However, this score is within the expected levels, as it includes teachers who use digital technologies somewhat regularly. In different contexts, the results show that there are still some difficulties in defining which technologies are better suited to different situations and how they can enhance the educational process. These results indeed indicate that the participants have digital competences, but are still at an evolutionary stage between digital literacy and digital proficiency and need, above all, to understand how to make use of digital technologies at different moments of the educational process. The Figure 2 shows that level B1 is identified as the highest number (43.1%). However, some teachers are still at levels A1 (2.1%) and A2 (18.4%), meaning that about 20% of the teachers who took part in this study have very limited digital competences. Consequently, it is very important to invest in their continuing training, especially in the areas where they have the greatest difficulties.
The analysis of the six areas of competence in this questionnaire and the results obtained (Table 4) shows that Area 1 -Professional engagement -is the one closest to Level B2 (see Table 3), and Area Table 4 -Assessmentlike the one with the weakest average result, still in Level A2, although it comes close to level B1.
To better understand these results, it is important to analyze the 21 items that form the questionnaire and justify the results shown in Table 4.  The analysis of Table 5 shows that the average results of Area 1 -Professional engagement -are always positive (more than 2.00 points), showing that these teachers are interested in reflecting on and learning more about the use of digital competences in educational environments. The items relating to this area are mostly related to the commitment to developing their digital competences either through online training or through the active search for different educational resources and teaching strategies that use the digital to enhance the educational process.
In line with the definition above of the instrument for level B1, in which the participants were placed, these results show their desire to know more and to use the digital in educational environments, recognizing the importance of knowing more resources, seeking more solutions and strategies to help them enhance their teaching and working processes in a school environment.
However, some frailties also exist, for example, in Area 4 -Assessment -, the one with the most problems, for which all items had weaker scores (less than 1.99 points), showing some difficulties in how the digital technologies are used in students' assessment. This area focuses on teachers' ability to use digital technologies to monitor their learning independently, and teachers can monitor their students' progress and give feedback to them.
Moreover, it is important to understand that other items also showed some weaknesses that require attention. Table 5 shows, in addition to the items corresponding to Area 4 -Assessment -, the items 'Guidance' in Area 3 -Teaching and Learning -and 'Communication', in Area 6 -Facilitating learners' digital competence -, which also presents lower scores. These items refer to using digital technologies to enhance students' autonomous work and help them communicate digitally with each other and with an external audience.
These items that concern the more practical work with digital technologies for the promotion of learning and content development, adapted to the individual needs of students, together with the promotion of the student's digital competences, are the ones that show the greatest difficulties and the weakest results among the participants of this study. Although the average result of Area 5 -Empowering learners -is also close to level B2, this area refers mostly to the teachers' reflections on their ability to monitor and follow-up on the students' progress and prepare activities digitally inclusive for all. However, whereas teachers indicate that they can have this reflective stance, the results in Area 4 -Assessment show that they have greater difficulty using digital technologies to assess students and, subsequently, provide them with the necessary feedback on the results.
In short, in theory, and according to the definition for level B1 -Integrator -, these results show the predisposition for using digital technologies and the recognition that they are valid for the educational process. At the same time, participants recognize that there is still much to be learned about using digital technologies in practice regarding the work to be developed in all teaching and learning processes.
Although studies indicate that age is not a relevant variable when it comes to assessing digital competences (Wang, Myers, & Sundaram, 2013;Dias-Trindade & Moreira, 2020), results by age show that teachers up to 40 years old present higher results of digital proficiency, and even average values in B2 -Expert -, which indicates that they are the most competent in the use of digital technologies in the areas under question.
As the age bracket increased, the result confirmed this idea, so did the average results decrease (Figure 3). This difference, however, is so small that it has no statistical significance (Table 6).  The analysis of the results by disciplinary department (Figure 4) was also analyzed, showing that the departments with the highest score were the Social and Human Sciences Department (43.7 points) and the Expressions Department (43.6 points), in line with the results of similar studies (Dias-Trindade & Moreira, 2020;Benali, Kaddouri, & Azzimani, 2018). However, once again, the difference in results has no statistical significance (Table 7).   Wang et al. (2013) pointed out, the issue of digital proficiency is not related to age or gender. In their study, these authors concluded that different factors influence the existence of more or less digital competence, such as social, psychological, organizational and even demographic factors, and that age is also one of them, albeit not the only one. More importantly, teachers are aware of the importance of digital competence in their profession to improve and enhance the educational process.
Therefore, it is important to understand that, according to the results achieved in this study, the participants are interested in and willing to integrate digital technologies in their practices. However, and given the world education situation today, in which different countries have been forced to shift to a completely digital context due to the confinement imposed to tackle the COVID-19 pandemic, a rapid diagnosis of the teachers' digital skills needs to be done to support the training actions to be developed.
In March 2020, the Global Education Innovation Initiative at the Harvard Graduate School of Education and the OECD drafted the survey 'Framework for Rapid Response to COVID-19', which was answered by respondents from 98 countries. Questioned whether there was any positive aspect resulting from the changes needed to adapt to the ongoing pandemic, 37.5% considered the issue of "[...] introduction of technologies and other innovative solutions [...]" to be very relevant (Reimers & Schleicher, 2020, p. 18). To this recognition is added the awareness that much needs to be done regarding teacher training, for rethinking strategies and redefining the teachers' formative structures, not only in terms of continuous training, but also (perhaps now more than ever) in terms of initial teachers' training, and which can lead to international public policies aimed at the development of a quality digital education perfectly suited to present and future needs.

Conclusion
Foundational literacy related to the ability to apply the knowledge acquired is no longer enough. It is necessary to articulate these competences with the ability to live in adaptive contexts, know how to work in groups in a cooperative and collaborative way, and know how to interact with the environment and any challenges that may arise.
By appropriating the potential of technologies, one can create innovative environments, sustainable learning scenarios where the digital can be an open window to the world of knowledge. However, for this to happen, teachers (and students) urgently need to recognize the value of these resources and, above all, to identify within digital technologies ways of adding value to the educational processes. More than having the ability to 'know how to do' something, being effectively 'competent to' may not be as simple as it seems.
Teachers also need to be able to help their students to make use of technologies in their education. Although various digital technologies always surround many students, they are only familiar with their social and non-school use. To shift to an educational context is still not a normal practice. It is precisely the heart of the challenge: bridging the gap to the educational world, recognizing the differences concerning specific uses, and taking advantage of these resources' potential.
The school can and should be a space where quality learning is developed, when the continuous flow of information, in increasingly large amounts, lacks appropriate competences and the ability to make the best use of digital technology. However, as can be seen from this study's results, the level of digital proficiency of teachers in Portugal differs widely and shows the different needs in various areas of competence.
This study is so important, as it provided a framework of the needs and competences of Portuguese teachers before the onset of the COVID-19 pandemic, and gave the opportunity to immediately adapt the most needed training, at a time when the use of digital environments has enabled the educational activities to continue.
We were able to conclude that teachers in Portugal are at a medium level (B1 -Integrator), showing their motivation to use the digital in their teaching processes, while also being aware of their weaknesses when it comes to practical work with students, monitoring them in their learning and giving regular feedback.
Level B1 -Integrator -characterizes teachers as professionals who use digital technologies in different contexts and are willing to do more, but they still need to know which technologies work better in each strategy and teaching method. Two key issues arise in close connection with the results obtained in this study and with the notion of digital proficiency: the ability of teachers to take advantage of the different potentialities of digital technologies available to teaching and learning, and the ability to know not only which technologies can be used more, but also which ones can best help to attain the objectives.
It is necessary to invest in teacher training aimed at practical work with students, as the areas showing the greatest weaknesses are Area 4 -Assessment -and also Area 6 -Facilitating learners' digital competence -, in particular the items that concern practical work, teaching, assessment, monitoring of students and also helping the development of the students' digital competences.
Although they are not statistically significant, we also concluded that the levels of digital proficiency are higher among younger teachers, and that the Department of Social and Human Sciences shows the best results.
However, it should be noted that teachers up to 40 years old present average overall results at the level of B2 -Expert-, revealing the critical and creative use of digital technologies in their teaching. Even so, continuous investment is needed in specific areas, and the development of the process towards digital proficiency is always in progress. Being aware of the continuing need for training and adaptation to digital technologies' growth is important and necessary not to run the risk of stagnation.
In view of these results, and taking into account the current state of education, as the result of the confinement measures across the world, the physical closure of schools, and the need to continue with the education process in exclusively digital environments, rethinking the training strategies regarding digital environments at school is that much more relevant, as is the fact that teachers should be able to use digital technologies to enhance their educational strategies.