CN-116097120-B - Display method and display device

CN116097120BCN 116097120 BCN116097120 BCN 116097120BCN-116097120-B

Abstract

The application discloses a display method and display equipment, wherein a camera can rotate within a preset angle range, a controller is configured to acquire character sound source information acquired by a sound acquisition device and conduct sound source identification, sound source angle information for identifying the azimuth angle of a position where a character is located is determined, a target rotation direction and a target rotation angle of the camera are determined based on the current shooting angle and the sound source angle information of the camera, and the shooting angle of the camera is adjusted according to the target rotation direction and the target rotation angle so that the shooting area of the camera is opposite to the position where the character is located when the voice is.

Inventors

YANG LUMING
WANG DAYONG
WANG XUSHENG
CHENG JIN
YU WENQIN
MA LE
DING JIAYI

Assignees

海信视像科技股份有限公司

Dates

Publication Date: 20260512
Application Date: 20210513
Priority Date: 20200701

Claims (10)

1. A display device, characterized by comprising: a display configured to present one or more images and one or more user interfaces, wherein the one or more images comprise images obtained from a broadcast system or network; an interface assembly configured to connect the camera and the sound collection assembly, the camera rotatable about a photographing angle configured to The sound collection assembly comprises a microphone array consisting of a plurality of microphones and is configured to collect audio signals; The controller is configured to start to acquire a test audio signal input by a user when the image shot by the camera does not contain a portrait; Locating a target position responsive to the test audio signal, the target position being acquired in accordance with the sound acquisition component The method comprises the steps of obtaining a test audio signal time difference of a camera, sending a rotation instruction to the camera to adjust the shooting direction of the camera to the target azimuth, obtaining a proofreading image, stopping obtaining test audio input by a user again until the image shot by the camera contains a portrait pattern, and generating a tracking instruction according to the position of the portrait pattern in the proofreading image, wherein the position of the portrait pattern in the proofreading image is determined according to skeleton line patterns established by a plurality of key points identified in the proofreading image; Detecting a portrait pattern in the proofreading image, and determining a preset area, wherein the preset area sets a maximum allowable coordinate error based on the central position of the portrait; If the portrait pattern is in the preset area, the shooting direction of the camera is maintained; and if the portrait pattern is not in the preset area, responding to the tracking instruction to adjust the shooting direction of the camera.
2. The display device according to claim 1, wherein the controller performs acquisition of the proof image until the portrait pattern is included in the image captured by the camera is further configured to: Acquiring a proofreading image through the camera; identifying at least one key point in the proofreading image, and establishing a skeleton line graph according to the identified key point; and determining a portrait position according to the skeleton line graph, marking the portrait position, sending a tracking instruction to the camera when the user moves the position, and adjusting the shooting direction of the camera according to the portrait position so as to track the position of the user.
3. The display device of claim 2, wherein the controller performs the step of moving the position toward the user The camera sends a tracking instruction, adjusts the shooting direction of the camera according to the portrait position so as to track the user position, and is further configured to track the user position according to the following steps: Acquiring a proofreading image through a camera according to a set frequency; detecting the position of a portrait pattern in the proofreading image; and if the portrait pattern is not in the preset area, generating a tracking instruction according to the portrait pattern position, wherein the tracking instruction comprises a rotation direction and a rotation angle, and transmitting the tracking instruction to the camera.
4. The display device according to claim 2, wherein the controller performs marking of the portrait position In the step, if the proof image includes a plurality of portrait patterns, the method is further configured to: searching a portrait pattern positioned in the center area of the proofreading image; If the center area of the proofreading image contains the portrait pattern, marking the portrait position corresponding to the portrait pattern in the center area of the image; and if the center area position of the proofreading image does not contain the portrait pattern, marking the portrait position corresponding to the portrait pattern with the largest area in the proofreading image.
5. The display device of claim 1, wherein the controller performs sending a rotation to the camera Instruction-turning, further configured to: acquiring an initial image through the camera; identifying a portrait pattern in the initial image; if the initial image contains the portrait pattern, sending a rotation instruction to the camera; And if the initial image does not contain the portrait pattern, acquiring a test audio signal which is input by the user again and used for executing the person positioning.
6. The display device according to claim 1, wherein the controller performs, in response to the tracking instruction, adjustment of the shooting direction of the camera if the portrait pattern is not within a preset area, and is further configured to: acquiring a skeleton line graph in a multi-frame correction image; Identifying a user movement state according to the bone line pattern; and calculating a motion change rule according to the motion state corresponding to the multi-frame correction image, and dynamically adjusting the shooting direction of the camera according to the motion change rule.
7. The display device according to claim 1, wherein the controller performs, when no portrait is included in the image captured by the camera, the acquisition of the test audio signal input by the user is started, and is further configured to: acquiring a sound signal through the sound acquisition component; extracting voiceprint information from the sound signal; Comparing the voiceprint information with a preset test voiceprint; If the voiceprint information is the same as a preset test voiceprint, marking the sound signal as a test audio signal; And if the voiceprint information is different from the preset test voiceprint, controlling the display to display a prompt interface.
8. The display device of claim 1, wherein the controller performs sending a rotation to the camera Instruction-turning, further configured to: acquiring a proofreading image and detecting a user position in the proofreading image; Comparing the portrait position with a preset judging area; If the portrait position is located in the preset judging area, a display is controlled to display the image shot by the camera in real time; if the portrait position is located outside the preset judging area, calculating the coordinate difference between the portrait position and the center of the preset judging area, generating a rotation instruction according to the coordinate difference, and sending the rotation instruction to the camera.
9. A display device, characterized by comprising: a display configured to present one or more images and one or more user interfaces, wherein the one or more images comprise images obtained from a broadcast system or network; a camera rotatable by a photographing angle configured to photograph an image; a sound collection assembly including a microphone array of a plurality of microphones configured to collect audio signals; A controller configured to: when the image shot by the camera does not contain a portrait, starting to acquire a test audio signal input by a user; Responding to the test audio signal, positioning a target azimuth, wherein the target azimuth is obtained by calculation according to the time difference of the test audio signal acquired by the sound acquisition component; sending a rotation instruction to the camera to adjust the shooting direction of the camera to the target azimuth; Acquiring a proofreading image, stopping acquiring test audio input by a user again until the image shot by the camera contains a portrait pattern, and generating a tracking instruction according to the position of the portrait pattern in the proofreading image, wherein the portrait pattern is a portrait pattern The position of the pattern in the proofreading image is determined according to skeleton line patterns established by a plurality of key points identified in the proofreading image; Detecting a portrait pattern in the proofreading image, and determining a preset area, wherein the preset area sets a maximum allowable coordinate error based on the central position of the portrait; if the portrait pattern is in the preset area, keeping the shooting direction of the camera unchanged; if the portrait pattern is not in the preset area, responding to the tracking instruction to adjust the shooting direction of the camera And (3) finishing.
10. A sound image person localization tracking method, characterized by being applied to a display device comprising a display and a controller, the display being configured to present one or more images and one or more user interfaces, wherein the one or more images comprise images obtained from a broadcast system or a network; The display device is internally provided with or externally connected with a camera and a sound acquisition assembly through an interface assembly, the camera can rotate a shooting angle, and the sound image character positioning and tracking method comprises the following steps: when the image shot by the camera does not contain a portrait, starting to acquire a test audio signal input by a user; Responding to the test audio signal, positioning a target azimuth, wherein the target azimuth is obtained by calculation according to the time difference of the test audio signal acquired by the sound acquisition component; sending a rotation instruction to the camera to adjust the shooting direction of the camera to the target azimuth; Acquiring a proofreading image, stopping acquiring test audio input by a user again until the image shot by the camera contains a portrait pattern, and generating a tracking instruction according to the position of the portrait pattern in the proofreading image, wherein the position of the portrait pattern in the proofreading image is determined according to skeleton line patterns established by a plurality of key points identified in the proofreading image; Detecting a portrait pattern in the proofreading image, and determining a preset area, wherein the preset area sets a maximum allowable coordinate error based on the central position of the portrait; if the portrait pattern is in the preset area, keeping the shooting direction of the camera unchanged; and if the portrait pattern is not in the preset area, responding to the tracking instruction to adjust the shooting direction of the camera.

Description

Display method and display device The application claims priority of Chinese patent application with application number 202010848905.X and name of "a sound image character positioning tracking method" filed in 21 st 8 th 2020, priority of Chinese patent application with application number 202010621070.4 and name of "a camera shooting angle adjusting method and display device" filed in 7 th 2020, and the whole content is incorporated by reference, and priority of Chinese patent application with application number 202110014128.3 and name of "a display device and sound image character positioning tracking method" filed in1 st 2021 month 6. Technical Field The application relates to the technical field of television software, in particular to a display method and display equipment. Background With the rapid development of display devices, the functions of the display devices are more and more abundant, and the performances of the display devices are more and more powerful. For example, the display device may implement web search, IP television, BBTV video on demand, video On Demand (VOD), digital music, web news, web video telephony, and the like. When the display equipment is used for realizing the network video call function, a camera is required to be installed on the display equipment, so that the acquisition of the user image is realized. Disclosure of Invention An embodiment of the present application provides a display apparatus including: the camera is configured to collect a portrait and realize rotation within a preset angle range; the sound collector is configured to collect character sound source information, wherein the character sound source information refers to sound information generated when a character interacts with the display device through voice; The controller is connected with the camera and the sound collector and is configured to acquire the character sound source information acquired by the sound collector and the current shooting angle of the camera; performing sound source identification on the character sound source information, and determining sound source angle information, wherein the sound source angle information is used for representing the azimuth angle of the position of the character in the process of voice; determining a target rotation direction and a target rotation angle of the camera based on the current shooting angle and the sound source angle information of the camera; And adjusting the shooting angle of the camera according to the target rotating direction and the target rotating angle so that the shooting area of the camera is opposite to the position where the person is in voice. Drawings In order to more clearly illustrate the technical solution of the present application, the drawings that are needed in the embodiments will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort. A schematic diagram of an operational scenario between a display device and a control apparatus according to some embodiments is schematically shown in fig. 1; A hardware configuration block diagram of a display device 200 according to some embodiments is exemplarily shown in fig. 2; A hardware configuration block diagram of the control device 100 according to some embodiments is exemplarily shown in fig. 3; A schematic diagram of the software configuration in a display device 200 according to some embodiments is exemplarily shown in fig. 4; An icon control interface display schematic of an application in a display device 200 according to some embodiments is illustrated in fig. 5; a block diagram of a display device according to some embodiments is shown schematically in fig. 6; a schematic diagram of a preset angular range to achieve camera rotation is shown schematically in fig. 7 according to some embodiments; A scene graph of camera rotation within a preset angular range is illustrated in fig. 8, in accordance with some embodiments; a schematic diagram of a sound source angular range according to some embodiments is exemplarily shown in fig. 9; A flowchart of a method of adjusting a camera shooting angle according to some embodiments is exemplarily shown in fig. 10; a flow chart of a comparative method of waking up text according to some embodiments is illustrated in fig. 11; A method flow diagram for sound source identification of character sound source information according to some embodiments is illustrated in fig. 12; a method flow diagram for determining a target rotational direction and a target rotational angle of a camera according to some embodiments is illustrated in fig. 13; A scene graph for adjusting camera shooting angle according to some embodiments is illustrated in fig. 14; another scene graph for adjusting camera shooting angle according to some embodiments is illustrated in fig. 15 a; A scene graph of where a person is located when speaking is shown schematic